tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.hcKsdUOEBx', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 45.79 seconds) gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for unit setup-gitlab.service gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab: waiting for the VM to finish booting gitlab # Filesystem UUID: e277d0e7-f316-4ec7-9cca-88fc846c3d38 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0kiiwgrdpq7n27p9r60rkrigahq0wnnq-nixos-system-gitlab-test/init regInfo=/nix/store/kg98b7gy56a0d1h0yviw6my9vchv4r8a-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 480632199 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor gitlab # [ 0.000809] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000878] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002730] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002741] Using GB pages for direct mapping gitlab # [ 0.002852] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002859] ACPI: Early table checksum verification disabled gitlab # [ 0.002862] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002866] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002869] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002876] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002878] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002880] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002881] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002883] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002884] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002885] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002886] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002886] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002887] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002887] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003116] No NUMA configuration found gitlab # [ 0.003117] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003120] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.005492] Zone ranges: gitlab # [ 0.005493] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005494] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005496] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005497] Device empty gitlab # [ 0.005497] Movable zone start for each node gitlab # [ 0.005498] Early memory node ranges gitlab # [ 0.005498] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005499] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005500] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005501] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005522] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005794] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.116136] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.116579] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.116596] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.116631] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.116635] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.116636] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.116638] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.116639] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.116640] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.116642] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.116644] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.116649] TSC deadline timer available gitlab # [ 0.116654] CPU topo: Max. logical packages: 1 gitlab # [ 0.116655] CPU topo: Max. logical dies: 1 gitlab # [ 0.116655] CPU topo: Max. dies per package: 1 gitlab # [ 0.116659] CPU topo: Max. threads per core: 1 gitlab # [ 0.116659] CPU topo: Num. cores per package: 4 gitlab # [ 0.116660] CPU topo: Num. threads per package: 4 gitlab # [ 0.116660] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.116686] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.116705] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.116708] kvm-guest: setup PV sched yield gitlab # [ 0.116725] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.116727] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.116728] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.116729] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.116731] Booting paravirtualized kernel on KVM gitlab # [ 0.116735] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.121199] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.123489] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.123541] kvm-guest: PV spinlocks enabled gitlab # [ 0.123543] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.123546] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0kiiwgrdpq7n27p9r60rkrigahq0wnnq-nixos-system-gitlab-test/init regInfo=/nix/store/kg98b7gy56a0d1h0yviw6my9vchv4r8a-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.123646] Unknown kernel command line parameters "regInfo=/nix/store/kg98b7gy56a0d1h0yviw6my9vchv4r8a-closure-info/registration", will be passed to user space. gitlab # [ 0.123659] random: crng init done gitlab # [ 0.123659] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.132675] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.137302] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.137384] software IO TLB: area num 4. gitlab # [ 0.214141] Fallback order for Node 0: 0 gitlab # [ 0.214149] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.214151] Policy zone: Normal gitlab # [ 0.216732] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.227094] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.241238] allocated 12582912 bytes of page_ext gitlab # [ 0.251894] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.251896] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.252782] Dynamic Preempt: lazy gitlab # [ 0.252976] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.252976] rcu: RCU event tracing is enabled. gitlab # [ 0.252977] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.252979] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.252979] Rude variant of Tasks RCU enabled. gitlab # [ 0.252980] Tracing variant of Tasks RCU enabled. gitlab # [ 0.252981] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.252981] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.252997] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.252999] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.253000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.257408] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.257737] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.257747] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.257864] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.261459] Console: colour VGA+ 80x25 gitlab # [ 0.261462] printk: legacy console [tty0] enabled gitlab # [ 0.292142] printk: legacy console [ttyS0] enabled gitlab # [ 0.400891] ACPI: Core revision 20250807 gitlab # [ 0.401840] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.403491] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.404550] x2apic enabled gitlab # [ 0.405310] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.406259] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.407530] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.409103] kvm-guest: setup PV IPIs gitlab # [ 0.410733] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.411802] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.413661] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) gitlab # [ 0.414745] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.415823] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.417658] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.418662] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.420659] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.421658] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.422658] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.423657] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.424658] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.425667] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.426659] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.427659] active return thunk: srso_alias_return_thunk gitlab # [ 0.428676] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.429659] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.430658] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.431658] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.432658] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.433658] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.435556] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.436658] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.437658] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.438658] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.439658] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.440658] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.441658] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.472319] Freeing SMP alternatives memory: 44K gitlab # [ 0.472661] pid_max: default: 32768 minimum: 301 gitlab # [ 0.473782] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.474807] landlock: Up and running. gitlab # [ 0.475659] Yama: becoming mindful. gitlab # [ 0.476878] LSM support for eBPF active gitlab # [ 0.477895] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.478809] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.482062] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.483240] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.483668] ... version: 2 gitlab # [ 0.484491] ... bit width: 48 gitlab # [ 0.485259] ... generic counters: 6 gitlab # [ 0.485660] ... generic bitmap: 000000000000003f gitlab # [ 0.486623] ... fixed-purpose counters: 0 gitlab # [ 0.487225] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.487661] ... value mask: 0000ffffffffffff gitlab # [ 0.488580] ... max period: 00007fffffffffff gitlab # [ 0.489377] ... global_ctrl mask: 000000000000003f gitlab # [ 0.489780] signal: max sigframe size: 3376 gitlab # [ 0.490617] rcu: Hierarchical SRCU implementation. gitlab # [ 0.491310] rcu: Max phase no-delay instances is 400. gitlab # [ 0.491863] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.497211] smp: Bringing up secondary CPUs ... gitlab # [ 0.498404] smpboot: x86: Booting SMP configuration: gitlab # [ 0.498665] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.501759] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.503446] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) gitlab # [ 0.504883] Memory: 6011604K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved) gitlab # [ 0.506660] devtmpfs: initialized gitlab # [ 0.507704] x86/mm: Memory block size: 128MB gitlab # [ 0.510850] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.512741] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.513763] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.515020] PM: RTC time: 16:13:09, date: 2026-09-06 gitlab # [ 0.518739] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.521022] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.521723] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.523797] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.524673] audit: initializing netlink subsys (disabled) gitlab # [ 0.525716] audit: type=2000 audit(1788711189.472:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.525917] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.527661] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.528661] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.529661] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.530660] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.531695] cpuidle: using governor menu gitlab # [ 0.533893] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.534923] PCI: Using configuration type 1 for base access gitlab # [ 0.535662] PCI: Using configuration type 1 for extended access gitlab # [ 0.536839] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.539684] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.540661] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.541661] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.542661] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.547439] ACPI: Added _OSI(Module Device) gitlab # [ 0.549667] ACPI: Added _OSI(Processor Device) gitlab # [ 0.551669] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.558729] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.565210] ACPI: Interpreter enabled gitlab # [ 0.567702] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.568666] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.570750] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.574668] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.578025] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.590845] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.592674] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.596302] acpiphp: Slot [3] registered gitlab # [ 0.597672] acpiphp: Slot [4] registered gitlab # [ 0.598704] acpiphp: Slot [5] registered gitlab # [ 0.600723] acpiphp: Slot [6] registered gitlab # [ 0.601706] acpiphp: Slot [7] registered gitlab # [ 0.602705] acpiphp: Slot [8] registered gitlab # [ 0.604739] acpiphp: Slot [9] registered gitlab # [ 0.605707] acpiphp: Slot [10] registered gitlab # [ 0.607704] acpiphp: Slot [11] registered gitlab # [ 0.608704] acpiphp: Slot [12] registered gitlab # [ 0.609722] acpiphp: Slot [13] registered gitlab # [ 0.611704] acpiphp: Slot [14] registered gitlab # [ 0.612703] acpiphp: Slot [15] registered gitlab # [ 0.613388] acpiphp: Slot [16] registered gitlab # [ 0.613689] acpiphp: Slot [17] registered gitlab # [ 0.614394] acpiphp: Slot [18] registered gitlab # [ 0.615681] acpiphp: Slot [19] registered gitlab # [ 0.616352] acpiphp: Slot [20] registered gitlab # [ 0.616690] acpiphp: Slot [21] registered gitlab # [ 0.617406] acpiphp: Slot [22] registered gitlab # [ 0.618681] acpiphp: Slot [23] registered gitlab # [ 0.619362] acpiphp: Slot [24] registered gitlab # [ 0.619682] acpiphp: Slot [25] registered gitlab # [ 0.620353] acpiphp: Slot [26] registered gitlab # [ 0.620679] acpiphp: Slot [27] registered gitlab # [ 0.621403] acpiphp: Slot [28] registered gitlab # [ 0.622681] acpiphp: Slot [29] registered gitlab # [ 0.623352] acpiphp: Slot [30] registered gitlab # [ 0.623680] acpiphp: Slot [31] registered gitlab # [ 0.624430] PCI host bridge to bus 0000:00 gitlab # [ 0.625669] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.626661] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.627670] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.630669] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.633667] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.636696] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.639668] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.645040] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.648805] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.655763] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.657734] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.660667] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.662666] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.665666] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.669131] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.675881] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.679771] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.683230] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.685678] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.688125] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.703811] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.706701] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.709721] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.712775] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.718737] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.729686] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.732129] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.734720] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.762682] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.768221] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.779705] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.781690] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.784732] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.791081] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.801703] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.803682] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.805719] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.812857] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.820684] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.822700] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.824710] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.830497] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.837678] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.838668] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.840683] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.843049] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.848697] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.849564] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.850683] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.853934] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.857700] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.858617] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.859470] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.864697] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.869512] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.876704] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.878709] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.884188] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.888587] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.889388] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.890682] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.892930] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.897701] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.899374] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.900708] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.910050] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.913093] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.916007] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.918001] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.920853] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.926682] iommu: Default domain type: Translated gitlab # [ 0.928665] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.931678] ACPI: bus type USB registered gitlab # [ 0.933779] usbcore: registered new interface driver usbfs gitlab # [ 0.935742] usbcore: registered new interface driver hub gitlab # [ 0.937717] usbcore: registered new device driver usb gitlab # [ 0.942106] NetLabel: Initializing gitlab # [ 0.943667] NetLabel: domain hash size = 128 gitlab # [ 0.945664] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.947731] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.949680] PCI: Using ACPI for IRQ routing gitlab # [ 0.951812] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.952657] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.952657] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.957671] vgaarb: loaded gitlab # [ 0.959034] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.960664] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.968661] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.974681] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.976513] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.980126] pnp: PnP ACPI init gitlab # [ 0.982945] pnp: PnP ACPI: found 6 devices gitlab # [ 0.996630] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 1.000327] clocksource: Switched to clocksource acpi_pm gitlab # [ 1.002897] NET: Registered PF_INET protocol family gitlab # [ 1.008524] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 1.039062] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 1.040603] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 1.041978] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 1.045799] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 1.047165] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 1.048802] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 1.050429] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.051894] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.053339] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 1.054348] NET: Registered PF_XDP protocol family gitlab # [ 1.055244] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.056307] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.057373] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.058530] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.059722] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.061154] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.062183] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.064595] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.067505] PCI: CLS 0 bytes, default 64 gitlab # [ 1.068356] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.068801] Trying to unpack rootfs image as initramfs... gitlab # [ 1.069465] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 1.069587] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 1.092799] Initialise system trusted keyrings gitlab # [ 1.094066] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.105484] Key type asymmetric registered gitlab # [ 1.106265] Asymmetric key parser 'x509' registered gitlab # [ 1.107195] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.108752] io scheduler mq-deadline registered gitlab # [ 1.109588] io scheduler kyber registered gitlab # [ 1.113504] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.114817] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.117327] Linux agpgart interface v0.103 gitlab # [ 1.118127] ACPI: bus type drm_connector registered gitlab # [ 1.120553] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.121685] usbserial: USB Serial support registered for generic gitlab # [ 1.122727] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.122727] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.125079] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.126482] drop_monitor: Initializing network drop monitor service gitlab # [ 1.127711] NET: Registered PF_INET6 protocol family gitlab # [ 1.129715] Segment Routing with IPv6 gitlab # [ 1.130447] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.132355] IPI shorthand broadcast: enabled gitlab # [ 1.136670] sched_clock: Marking stable (988014891, 147910661)->(1160271779, -24346227) gitlab # [ 1.138477] registered taskstats version 1 gitlab # [ 1.139525] Loading compiled-in X.509 certificates gitlab # [ 1.149138] Demotion targets for Node 0: null gitlab # [ 1.150208] Key type .fscrypt registered gitlab # [ 1.150898] Key type fscrypt-provisioning registered gitlab # [ 1.151965] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.152897] ima: Allocated hash algorithm: sha1 gitlab # [ 1.153751] ima: No architecture policies found gitlab # [ 1.154848] PM: Magic number: 10:334:236 gitlab # [ 1.156663] RAS: Correctable Errors collector initialized. gitlab # [ 1.161265] clk: Disabling unused clocks gitlab # [ 1.161985] PM: genpd: Disabling unused power domains gitlab # [ 1.355802] Freeing initrd memory: 29392K gitlab # [ 1.361201] Freeing unused decrypted memory: 2028K gitlab # [ 1.367172] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.369817] Write protecting the kernel read-only data: 32768k gitlab # [ 1.374736] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.378701] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.447743] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.448957] Run /init as init process gitlab # [ 1.461868] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.485700] fuse: init (API version 7.45) gitlab # [ 1.493882] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.504092] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.512606] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.569801] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.907100] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.948023] systemd[1]: Detected virtualization kvm. gitlab # [ 1.950185] systemd[1]: Detected architecture x86-64. gitlab # [ 1.952341] systemd[1]: Running in initrd. gitlab # [ 1.954876] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.957981] systemd[1]: Hostname set to . gitlab # [ 2.066288] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 2.103626] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.113680] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.114863] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.116333] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.117492] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.118617] systemd[1]: Reached target Path Units. gitlab # [ 2.119516] systemd[1]: Reached target Slice Units. gitlab # [ 2.120434] systemd[1]: Reached target Swaps. gitlab # [ 2.121274] systemd[1]: Reached target Timer Units. gitlab # [ 2.122268] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.123504] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.124671] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.125696] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.126738] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.127708] systemd[1]: Reached target Socket Units. gitlab # [ 2.129770] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.131994] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.134126] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.136883] systemd[1]: Starting Journal Service... gitlab # [ 2.141034] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.143272] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.145746] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.148769] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.152762] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.154845] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.156269] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.159137] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.172673] systemd-journald[91]: Collecting audit messages is disabled. gitlab # [ 2.181487] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.189062] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.190648] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.193190] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.203698] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.205341] netfs: FS-Cache loaded gitlab # [ 2.206050] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.207618] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.208964] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.210168] systemd[1]: Reached target Local File Systems. gitlab # [ 2.214851] 9pnet: Installing 9P2000 support gitlab # [ 2.223007] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.224572] systemd[1]: Started Journal Service. gitlab # [ 2.076784] systemd-modules-load[93]: Using 4 probe threads gitlab # [ 2.078796] systemd-modules-load[93]: Inserted module 'virtio_balloon' gitlab # [ 2.081465] systemd-modules-load[93]: Inserted module 'virtio_gpu' gitlab # [ 2.082522] systemd-modules-load[93]: Inserted module 'dm_mod' gitlab # [ 2.083501] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.084752] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.085851] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.086795] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.098662] systemd-udevd[104]: Using default interface naming scheme 'v261'. gitlab # [ 2.103303] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.111688] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.142939] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.143782] systemd[1]: Reached target System Initialization. gitlab # [ 2.144613] systemd[1]: Reached target Basic System. gitlab # [ 2.386256] SCSI subsystem initialized gitlab # [ 2.391502] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.393834] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.394353] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.394833] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.397853] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.398812] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.398999] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.399296] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.403275] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.404467] usb usb1: Product: UHCI Host Controller gitlab # [ 2.405366] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.406221] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.411490] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.414164] virtio_blk virtio5: [vdb] 11842968 512-byte logical blocks (6.06 GB/5.65 GiB) gitlab # [ 2.416817] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.418325] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.421630] hub 1-0:1.0: USB hub found gitlab # [ 2.422346] hub 1-0:1.0: 2 ports detected gitlab # [ 2.427266] scsi host0: ata_piix gitlab # [ 2.432129] scsi host1: ata_piix gitlab # [ 2.432761] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.433857] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.300982] (udev-worker)[124]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.306363] (udev-worker)[123]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.308806] (udev-worker)[123]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.311069] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.317547] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.318697] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.320848] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.324976] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.333984] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.336531] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.343700] systemd-fsck[158]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.347119] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.589404] ata2: found unknown device (class 0) gitlab # [ 2.592294] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.596327] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.619385] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.647012] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.653038] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.678616] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.817385] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.820426] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.823027] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.824640] usb 1-1: Manufacturer: QEMU gitlab # [ 2.826172] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.857854] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.871407] usbcore: registered new interface driver usbhid gitlab # [ 2.873633] usbhid: USB HID core driver gitlab # [ 2.883328] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.887407] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 3.002789] systemd[1]: Mounting /sysroot... gitlab # [ 3.299440] EXT4-fs (vda): mounted filesystem e277d0e7-f316-4ec7-9cca-88fc846c3d38 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.157764] systemd[1]: Mounted /sysroot. gitlab # [ 3.159473] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.161521] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.164325] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.166959] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.169307] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.176320] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.184168] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.335350] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.188874] systemd-sysroot-fstab-check[178]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.191214] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.192054] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.194169] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.195119] systemd[1]: Reload requested from client PID 178 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.196833] systemd[1]: Reloading... gitlab # [ 3.349309] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.256910] systemd[1]: Reloading finished in 60 ms. gitlab # [ 3.275764] systemd-sysroot-fstab-check[178]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.278766] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.280798] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.282793] systemd-sysroot-fstab-check[178]: Requesting swap.target/start/replace... gitlab # [ 3.285433] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.287516] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.289868] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.292309] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.302559] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.305279] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.004338] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.054563] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.056457] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.058721] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.060536] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.089401] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.092731] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.107738] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.109645] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.111866] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.143916] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.146212] systemd[1]: Stopped target Basic System. gitlab # [ 4.147980] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.149845] systemd[1]: Stopped target Path Units. gitlab # [ 4.151564] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.154190] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.158716] systemd[1]: Stopped target Slice Units. gitlab # [ 4.161322] systemd[1]: Stopped target Socket Units. gitlab # [ 4.163137] systemd[1]: Stopped target System Initialization. gitlab # [ 4.166462] systemd[1]: Stopped target Swaps. gitlab # [ 4.168155] systemd[1]: Stopped target Timer Units. gitlab # [ 4.169810] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.173193] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.175256] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.177682] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.179411] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.182917] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.184565] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.186373] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.188127] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.190104] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.191549] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.193798] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.195968] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.198063] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.199756] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.201293] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.203054] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.204381] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.205463] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.206771] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.208261] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.209345] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.210619] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.211793] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.213059] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.214284] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.215556] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.216621] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.217834] systemd[1]: Closed udev Control Socket. gitlab # [ 4.218828] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.219842] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.221173] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.222291] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.223545] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.224537] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.225543] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.236736] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.237835] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.239158] systemd[1]: Reached target Switch Root. gitlab # [ 4.241144] systemd[1]: Starting NixOS Activation... gitlab # [ 4.382235] initrd-nixos-activation-start[225]: booting system configuration /nix/store/0kiiwgrdpq7n27p9r60rkrigahq0wnnq-nixos-system-gitlab-test gitlab # [ 4.430544] initrd-nixos-activation-start[225]: running activation script... gitlab # [ 4.668672] initrd-nixos-activation-start[248]: setting up /etc... gitlab # [ 4.779466] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.782267] systemd[1]: Finished NixOS Activation. gitlab # [ 4.784401] systemd[1]: Starting Switch Root... gitlab # [ 4.821980] systemd[1]: Switching root. gitlab # [ 5.032815] systemd-journald[91]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.147852] NET: Registered PF_VSOCK protocol family gitlab # [ 5.518088] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.523619] systemd[1]: Detected virtualization kvm. gitlab # [ 5.525548] systemd[1]: Detected architecture x86-64. gitlab # [ 5.527579] systemd[1]: Detected first boot. gitlab # [ 5.530271] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.669671] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.767517] systemd[1]: Applying preset policy. gitlab # [ 5.896548] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.014639] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.016023] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.018496] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.020644] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.022066] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.023616] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.024553] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.025906] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.027019] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.027736] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 6.028515] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.029404] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.030270] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.031104] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.031981] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.032969] systemd[1]: Reached target Path Units. gitlab # [ 6.033692] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.034510] systemd[1]: Reached target Slice Units. gitlab # [ 6.035275] systemd[1]: Reached target Swaps. gitlab # [ 6.036692] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 6.038675] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.040142] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.041803] systemd[1]: Listening on Factory Reset Management. gitlab # [ 6.042741] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 6.045132] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.046482] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.048460] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.049730] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.050926] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.052251] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.055108] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.056180] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.057082] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 6.060460] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.062520] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.064751] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.068059] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.081509] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.082525] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 6.086971] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 6.087822] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.091470] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.106458] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.111015] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.112028] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.124064] systemd[1]: Starting Journal Service... gitlab # [ 6.128861] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.142023] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.148775] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.151503] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.160139] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.162441] systemd-journald[320]: Collecting audit messages is enabled. gitlab # [ 6.170980] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.171986] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.172938] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.173967] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.174984] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.175213] loop: module loaded gitlab # [ 6.175572] EXT4-fs (vda): re-mounted e277d0e7-f316-4ec7-9cca-88fc846c3d38. gitlab # [ 6.183303] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.035995] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 6.185286] systemd[1]: Started Journal Service. gitlab # [ 6.038034] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.039583] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.040909] systemd-modules-load[321]: Using 3 probe threads gitlab # [ 6.042073] systemd-modules-load[321]: Inserted module 'tls' gitlab # [ 6.043095] systemd-modules-load[321]: Inserted module 'loop' gitlab # [ 6.044232] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.045467] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.046505] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.047776] systemd-oomd[323]: No swap; memory pressure usage will be degraded gitlab # [ 6.049383] systemd[1]: Finished Load Kernel Modules. gitlab # [ 6.050257] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.051219] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.052104] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.053075] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.053947] systemd[1]: Starting Firewall... gitlab # [ 6.068219] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.069252] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.070886] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.075089] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.081696] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.083169] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.089114] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.091849] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.093567] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.242962] systemd-journald[320]: Received client request to flush runtime journal. gitlab # [ 6.140965] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.142507] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.143811] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.145188] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.146193] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.147195] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.148369] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.159750] systemd-udevd[359]: Using default interface naming scheme 'v261'. gitlab # [ 6.176637] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.191603] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.270686] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.300857] (udev-worker)[437]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.304311] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.326565] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.333200] systemd[1]: Finished Firewall. gitlab # [ 6.345592] (udev-worker)[430]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.347839] (udev-worker)[430]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.362485] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.363887] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.365644] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.367961] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.369745] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.371194] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.520902] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.523160] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.534245] Console: switching to colour dummy device 80x25 gitlab # [ 6.535825] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.535827] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.540630] ACPI: button: Power Button [PWRF] gitlab # [ 6.541855] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.543306] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.559565] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.561178] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.561737] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.563308] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.583006] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 6.595199] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.595836] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.597453] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.598791] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.609412] FDC 0 is a S82078B gitlab # [ 6.612314] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 6.612401] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.613957] rtc_cmos 00:05: setting system clock to 2026-09-06T16:13:15 UTC (1788711195) gitlab # [ 6.615050] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.615207] systemd-journald[320]: Time jumped backwards, rotating. gitlab # [ 6.628211] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.653213] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.518915] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.531347] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.532608] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.541361] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.547821] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.548886] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.549685] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.698831] ppdev: user-space parallel port driver gitlab # [ 6.799549] kvm_amd: TSC scaling supported gitlab # [ 6.651918] systemd-vconsole-setup[494]: 6.800560] kvm_amd: Nested Virtualization enabled gitlab # 1;39mConfiguration of first virt[ 6.801530] kvm_amd: Nested Paging enabled gitlab # ual console was skipped, ignorin[ 6.802251] kvm_amd: LBR virtualization supported gitlab # g remaining ones[ 6.802842] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # . gitlab # [ 6.803651] kvm_amd: Virtual GIF supported gitlab # [ 6.804122] kvm_amd: Virtual NMI enabled gitlab # [ 6.656426] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.842322] EDAC MC: Ver: 3.0.0 gitlab # [ 6.872261] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.895410] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.897157] systemd[1]: Reached target Local File Systems. gitlab # [ 6.899106] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.901820] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.904309] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.906419] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.909588] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.911936] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.961779] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.969537] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.978363] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.982446] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.985101] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.000773] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.003319] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.007281] systemd[1]: Starting Update is Completed... gitlab # [ 7.021808] systemd[1]: Finished Update is Completed. gitlab # [ 7.080580] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.082750] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.328625] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.330893] systemd[1]: Reached target System Initialization. gitlab # [ 7.332911] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.335348] systemd[1]: Started logrotate.timer. gitlab # [ 7.336989] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.339301] systemd[1]: Reached target Timer Units. gitlab # [ 7.341079] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.343263] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.349574] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.351942] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.354705] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.356736] systemd[1]: Reached target Socket Units. gitlab # [ 7.358569] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.394222] dbus-broker-launch[577]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.397490] dbus-broker-launch[577]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.400092] dbus-broker-launch[577]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.405214] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.419773] dbus-broker-launch[577]: Ready gitlab # [ 7.422643] systemd[1]: Reached target Basic System. gitlab # [ 7.424762] systemd[1]: Started backdoor.service. gitlab # [ 7.427497] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.429872] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.439125] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.443930] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.449426] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.453315] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.456664] systemd[1]: Starting resolvconf update... gitlab # [ 7.466372] logrotate[582]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.470111] logrotate[582]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.473953] logrotate[582]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.475068] systemd[1]: Finished Logrotate configuration check. gitlab # connecting to host... gitlab # [ 7.475943] logrotate[582]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.478137] logrotate[582]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.478747] logrotate[582]: Allocating hash table for state file, size 64 entries gitlab # [ 7.480463] logrotate[582]: Handling 4 logs gitlab # [ 7.481348] logrotate[582]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.482229] logrotate[582]: considering log /var/log/btmp gitlab # [ 7.483240] logrotate[582]: Creating new state gitlab # [ 7.489170] logrotate[582]: Now: 2026-09-06 16:13 gitlab # [ 7.489804] logrotate[582]: Last rotated at 2026-09-06 16:00 gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.490415] logrotate[582]: log does not need rotating (log has already been rotated) gitlab # [ 7.490895] logrotate[582]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab: connected to guest root shell gitlab # [ 7.491804] logrotate[582]: considering log /var/log/wtmp gitlab: (connecting took 8.13 seconds) gitlab # [ 7.492405] logrotate[582]: Creating new state gitlab: (finished: waiting for the VM to finish booting, in 8.13 seconds) gitlab # [ 7.492894] logrotate[582]: Now: 2026-09-06 16:13 gitlab # [ 7.493566] logrotate[582]: Last rotated at 2026-09-06 16:00 gitlab # [ 7.494679] logrotate[582]: log does not need rotating (log has already been rotated) gitlab # [ 7.495098] logrotate[582]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.495531] logrotate[582]: switching euid from 0 to 165 and egid from 0 to 165 (pid 582) gitlab # [ 7.495961] logrotate[582]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.496863] logrotate[582]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.497509] logrotate[582]: Creating new state gitlab # [ 7.497940] logrotate[582]: switching euid from 165 to 0 and egid from 165 to 0 (pid 582) gitlab # [ 7.498585] logrotate[582]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.499254] logrotate[582]: switching euid from 0 to 60 and egid from 0 to 60 (pid 582) gitlab # [ 7.499630] logrotate[582]: considering log /var/log/nginx/*.log gitlab # [ 7.500086] logrotate[582]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.500521] logrotate[582]: Creating new state gitlab # [ 7.500962] logrotate[582]: not running postrotate script, since no logs were rotated gitlab # [ 7.501561] logrotate[582]: switching euid from 60 to 0 and egid from 60 to 0 (pid 582) gitlab # [ 7.516723] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.522794] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.526351] nsncd[583]: Sep 06 16:13:16.559 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.527093] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.528216] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.528847] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.533791] systemd[1]: Starting User Login Management... gitlab # [ 7.545427] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.546252] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.547354] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.547894] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.550152] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.552326] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.553060] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.553525] systemd[1]: nscd.service: Consumed 14ms CPU time over 107ms wall clock time, 7.3M memory peak, 5.5M read from disk. gitlab # [ 7.560984] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.572926] systemd-logind[648]: New seat seat0. gitlab # [ 7.578140] systemd-logind[648]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.578929] systemd-logind[648]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.582652] systemd-logind[648]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.583200] systemd[1]: Started User Login Management. gitlab # [ 7.584283] systemd[1]: Starting linger-users.service... gitlab # [ 7.585680] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.586716] nsncd[672]: Sep 06 16:13:16.620 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.596795] systemd[1]: Finished resolvconf update. gitlab # [ 7.598779] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.600070] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.601154] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.603313] systemd[1]: Starting DHCP Client... gitlab # [ 7.604336] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.607417] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.609489] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.610614] systemd[1]: Finished linger-users.service. gitlab # [ 7.630410] network-addresses-eth1-start[699]: adding address 192.168.1.1/24... done gitlab # [ 7.635035] network-addresses-eth1-start[699]: adding address 2001:db8:1::1/64... done gitlab # [ 7.642433] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.673465] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.675102] systemd[1]: Reached target Network. gitlab # [ 7.678252] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.680124] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.682381] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.689990] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.692596] dhcpcd[716]: dhcpcd-10.3.2 starting gitlab # [ 7.695070] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.695889] dhcpcd[769]: dev: loaded udev gitlab # [ 7.848840] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.850310] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.715837] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.720055] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.723474] systemd[1]: Started Getty on tty1. gitlab # [ 7.726521] systemd[1]: Reached target Login Prompts. gitlab # [ 7.953247] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.961402] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.962766] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.964982] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.966428] cfg80211: failed to load regulatory.db gitlab # [ 7.827918] nginx-pre-start[790]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.828720] nginx-pre-start[790]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.834820] systemd[1]: Started Nginx Web Server. gitlab # [ 7.839448] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.840264] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.840621] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started gitlab # [ 7.840788] redis-server[788]: Configuration loaded gitlab # [ 7.845186] redis[788]: monotonic clock: POSIX clock_gettime gitlab # [ 7.847468] postgresql-pre-start[792]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.848263] postgresql-pre-start[792]: This user must also own the server process. gitlab # [ 7.850797] redis[788]: Running mode=standalone, port=0. gitlab # [ 7.852370] redis[788]: Server initialized gitlab # [ 8.000561] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.854329] redis[788]: Ready to accept connections unix gitlab # [ 7.855260] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.855843] postgresql-pre-start[792]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.856575] postgresql-pre-start[792]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.857219] postgresql-pre-start[792]: The default text search configuration will be set to "english". gitlab # [ 7.857762] postgresql-pre-start[792]: Data page checksums are enabled. gitlab # [ 7.858835] postgresql-pre-start[792]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.859522] postgresql-pre-start[792]: creating subdirectories ... ok gitlab # [ 7.860060] postgresql-pre-start[792]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.860614] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.861205] dhcpcd[769]: eth0: waiting for carrier gitlab # [ 7.862189] dhcpcd[769]: eth0: carrier acquired gitlab # [ 7.869714] dhcpcd[769]: DUID 00:01:00:01:32:30:4d:9c:52:54:00:12:34:56 gitlab # [ 7.870713] dhcpcd[769]: eth0: IAID 00:12:34:56 gitlab # [ 7.872137] dhcpcd[769]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.899279] postgresql-pre-start[792]: selecting default "max_connections" ... 100 gitlab # [ 7.948657] postgresql-pre-start[792]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.212308] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.355665] postgresql-pre-start[792]: selecting default time zone ... UTC gitlab # [ 8.358228] postgresql-pre-start[792]: creating configuration files ... ok gitlab # [ 8.395536] gitlab-workhorse[824]: time="2026-09-06T16:13:17Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.397235] gitlab-workhorse[824]: time="2026-09-06T16:13:17Z" level=info msg="Using redis/go-redis" gitlab # [ 8.398533] gitlab-workhorse[824]: time="2026-09-06T16:13:17Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.534837] postgresql-pre-start[792]: running bootstrap script ... ok gitlab # [ 8.982118] postgresql-pre-start[792]: performing post-bootstrap initialization ... ok gitlab # [ 9.280590] postgresql-pre-start[792]: syncing data to disk ... ok gitlab # [ 9.282941] postgresql-pre-start[792]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.286158] postgresql-pre-start[792]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.291276] postgresql-pre-start[792]: Success. You can now start the database server using: gitlab # [ 9.294161] postgresql-pre-start[792]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.401536] postgres[851]: [851] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.405655] postgres[851]: [851] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.408500] postgres[851]: [851] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.411284] postgres[851]: [851] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.421526] postgres[860]: [860] LOG: database system was shut down at 2026-09-06 16:13:18 GMT gitlab # [ 9.428558] postgres[851]: [851] LOG: database system is ready to accept connections gitlab # [ 9.438520] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.440638] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.480120] dhcpcd[769]: eth0: soliciting a DHCP lease gitlab # [ 9.640366] NET: Registered PF_PACKET protocol family gitlab # [ 9.508319] dhcpcd[769]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.513071] dhcpcd[769]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.628613] postgresql-setup-start[872]: CREATE DATABASE gitlab # [ 9.659478] postgresql-setup-start[887]: CREATE ROLE gitlab # [ 9.673719] postgresql-setup-start[893]: ALTER DATABASE gitlab # [ 9.694799] postgresql-setup-start[898]: CREATE ROLE gitlab # [ 9.696994] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.698861] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.700421] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.780639] gitlab-postgresql-start[907]: CREATE DATABASE gitlab # [ 9.802886] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[892]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.806033] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[892]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.825846] gitlab-postgresql-start[911]: CREATE EXTENSION gitlab # [ 9.853574] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.861243] systemd[1]: Started gitaly.service. gitlab # [ 9.863087] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.909113] gitlab-postgresql-start[920]: CREATE EXTENSION gitlab # [ 9.914546] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.009481] dhcpcd[769]: eth0: soliciting an IPv6 router gitlab # [ 10.011284] dhcpcd[769]: eth0: Router Advertisement from fe80::2 gitlab # [ 10.012964] dhcpcd[769]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 10.015358] dhcpcd[769]: eth0: adding route to fec0::/64 gitlab # [ 10.017510] dhcpcd[769]: eth0: adding default route via fe80::2 gitlab # [ 10.432027] gitaly[927]: time="2026-09-06T16:13:19.465Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=927 gitlab # [ 10.433217] gitaly[927]: time="2026-09-06T16:13:19.465Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=927 gitlab # [ 10.434690] gitaly[927]: time="2026-09-06T16:13:19.465Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=927 gitlab # [ 10.436138] gitaly[927]: time="2026-09-06T16:13:19.466Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=927 gitlab # [ 10.436897] gitaly[927]: time="2026-09-06T16:13:19.466Z" level=info msg="Starting Gitaly" pid=927 version= gitlab # [ 10.632060] gitaly[927]: time="2026-09-06T16:13:19.665Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=198 pid=927 gitlab # [ 10.632953] gitaly[927]: time="2026-09-06T16:13:19.665Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=927 gitlab # [ 10.634745] gitaly[927]: time="2026-09-06T16:13:19.666Z" level=info msg="finished initializing command factory" duration_ms=0 pid=927 gitlab # [ 10.635607] gitaly[927]: time="2026-09-06T16:13:19.666Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-927/git-exec-1600832631.d/git pid=927 gitlab # [ 10.639553] gitaly[927]: time="2026-09-06T16:13:19.673Z" level=info msg="finished detecting git version" duration_ms=6 pid=927 gitlab # [ 10.640584] gitaly[927]: time="2026-09-06T16:13:19.674Z" level=info msg="using Git version" pid=927 version=2.53.g gitlab # [ 10.641843] gitaly[927]: time="2026-09-06T16:13:19.675Z" level=info msg="clearing disk cache object folder" pid=927 storage=default gitlab # [ 10.642660] gitaly[927]: time="2026-09-06T16:13:19.676Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1481186889 pid=927 storage=default gitlab # [ 10.643650] gitaly[927]: time="2026-09-06T16:13:19.676Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1481186889 pid=927 storage=default gitlab # [ 10.644352] gitaly[927]: time="2026-09-06T16:13:19.676Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=927 gitlab # [ 10.644759] gitaly[927]: time="2026-09-06T16:13:19.676Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=927 gitlab # [ 10.645609] gitaly[927]: time="2026-09-06T16:13:19.676Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1481186889 clear_duration_ms=0 pid=927 storage=default gitlab # [ 10.696252] gitaly[927]: time="2026-09-06T16:13:19.729Z" level=info msg="counting repositories" pid=927 gitlab # [ 10.696856] gitaly[927]: time="2026-09-06T16:13:19.730Z" level=info msg="starting to count repositories" pid=927 storage_path=/var/gitlab/state/repositories gitlab # [ 10.697681] gitaly[927]: time="2026-09-06T16:13:19.729Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=927 gitlab # [ 10.698624] gitaly[927]: time="2026-09-06T16:13:19.730Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=927 storage_path=/var/gitlab/state/repositories gitlab # [ 10.699372] gitaly[927]: time="2026-09-06T16:13:19.730Z" level=info msg="completed counting all repositories" counting_duration=0 pid=927 gitlab # [ 10.699572] gitaly[927]: time="2026-09-06T16:13:19.730Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=927 schema=unix gitlab # [ 10.700184] gitaly[927]: time="2026-09-06T16:13:19.730Z" level=info msg="listening at address" address=/run/gitaly/gitaly-927/sock.d/intern pid=927 schema=unix gitlab # [ 10.700802] gitaly[927]: time="2026-09-06T16:13:19.733Z" level=info msg="finished tempdir cleaner walk" pid=927 storage=default time_ms=0 gitlab # [ 10.712516] gitaly[927]: time="2026-09-06T16:13:19.746Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=927 gitlab # [ 10.713403] gitaly[927]: time="2026-09-06T16:13:19.746Z" level=info msg="Started Gitaly" duration_ms=280 pid=927 gitlab # [ 10.713688] gitaly[927]: time="2026-09-06T16:13:19.746Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=927 gitlab # [ 10.717671] gitaly[927]: time="2026-09-06T16:13:19.747Z" level=info msg="maintenance: daily scheduled" pid=927 scheduled="2026-09-07 12:00:00 +0000 UTC" gitlab # [ 11.413306] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[930]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.414307] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[930]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.452818] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[930]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.684246] gitaly[927]: time="2026-09-06T16:13:21.716Z" level=info msg="License database preloaded" duration_ms=2223 pid=927 gitlab # [ 14.127038] dhcpcd[769]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.128104] dhcpcd[769]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.128859] dhcpcd[769]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.223389] systemd[1]: Started DHCP Client. gitlab # [ 14.224895] systemd[1]: Reached target Network is Online. gitlab # [ 42.009698] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[930]: W, [2026-09-06T16:13:51.042636 #930] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 44.239533] postgres[1317]: [1317] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 44.240707] postgres[1317]: [1317] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 44.241718] postgres[1317]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 44.242688] postgres[1317]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 44.243507] postgres[1317]: attidentity AS identity, gitlab # [ 44.244591] postgres[1317]: attgenerated as attgenerated gitlab # [ 44.245581] postgres[1317]: FROM pg_attribute a gitlab # [ 44.247117] postgres[1317]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 44.248646] postgres[1317]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 44.250237] postgres[1317]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 44.251655] postgres[1317]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 44.252714] postgres[1317]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 44.254408] postgres[1317]: ORDER BY a.attnum gitlab # [ 44.255722] postgres[1317]: gitlab # [ 47.685597] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1359]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 47.686882] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1359]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.451504] gitlab-workhorse[824]: time="2026-09-06T16:13:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.461450] gitlab-workhorse[824]: time="2026-09-06T16:14:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 63.458321] postgres[1425]: [1425] ERROR: relation "table_without_models" does not exist gitlab # [ 63.460628] postgres[1425]: [1425] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.462667] gitlab-workhorse[824]: time="2026-09-06T16:14:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 75.926639] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 75.931665] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 75.934835] systemd[1]: Starting gitlab.service... gitlab # [ 76.282124] bundle[1610]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 76.283473] bundle[1610]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 76.287321] bundle[1610]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 78.472101] gitlab-workhorse[824]: time="2026-09-06T16:14:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.070463] sidekiq-cluster[1609]: {"severity":"INFO","time":"2026-09-06T16:14:28.086Z","message":"Starting cluster with 1 processes"} gitlab # [ 79.147761] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.161Z","pid":1610,"message":"* Enabling systemd notification integration"} gitlab # [ 79.148887] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.180Z","pid":1610,"message":"Puma starting in cluster mode..."} gitlab # [ 79.149729] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.180Z","pid":1610,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 79.150551] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.180Z","pid":1610,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 79.151563] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.181Z","pid":1610,"message":"* Min threads: 0"} gitlab # [ 79.152550] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.181Z","pid":1610,"message":"* Max threads: 4"} gitlab # [ 79.153285] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.181Z","pid":1610,"message":"* Environment: production"} gitlab # [ 79.153860] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.181Z","pid":1610,"message":"* Master PID: 1610"} gitlab # [ 79.154771] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.181Z","pid":1610,"message":"* Workers: 2"} gitlab # [ 79.155482] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.181Z","pid":1610,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 79.156204] bundle[1610]: {"timestamp":"2026-09-06T16:14:28.181Z","pid":1610,"message":"* Preloading application"} gitlab # [ 88.474797] gitlab-workhorse[824]: time="2026-09-06T16:14:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.484101] gitlab-workhorse[824]: time="2026-09-06T16:14:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 101.967512] bundle[1610]: W, [2026-09-06T16:14:51.000511 #1610] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 104.408676] sidekiq-cluster[1640]: 2026-09-06T16:14:53.441Z pid=1640 tid=wjk INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1640, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>434278400} gitlab # [ 104.581274] sidekiq-cluster[1640]: W, [2026-09-06T16:14:53.614849 #1640] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 104.620788] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:53.654Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 106.488916] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.521Z","message":"GitLab reliable fetch activated!"} gitlab # [ 106.490254] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.521Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 106.510576] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.544Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 106.512780] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.546Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 106.515286] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.549Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 106.517596] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.551Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 106.519948] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.553Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 106.522333] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.556Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 106.524310] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.558Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 106.526146] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.559Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 106.528089] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.561Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 106.529978] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.563Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 106.531835] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.565Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 106.534310] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.568Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 106.536294] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.570Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 106.538229] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.572Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 106.540400] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.574Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 106.542675] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.576Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 106.545144] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.578Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 106.547783] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.581Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 106.550376] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.584Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 106.552980] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.586Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 106.555914] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.589Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 106.558496] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.592Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 106.560430] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.594Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 106.562642] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.596Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 106.564907] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.598Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 106.567506] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.601Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 106.570508] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.604Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 106.572957] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.606Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 106.575403] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.609Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 106.577819] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.611Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 106.580117] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.613Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 106.582497] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.616Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 106.586063] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.619Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 106.589132] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.622Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 106.592673] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.626Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 106.595698] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.629Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 106.598835] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.632Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 106.602437] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.636Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 106.605615] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.639Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 106.608548] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.642Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 106.611663] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.645Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 106.614267] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.648Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 106.617216] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.651Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 106.620241] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.654Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 106.623359] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.657Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 106.626211] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.660Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 106.628746] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.662Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 106.631667] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.665Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 106.634744] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.668Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 106.637721] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.671Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 106.640733] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.674Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 106.643860] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.677Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 106.653211] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.687Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 106.656540] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.690Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 106.659659] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.693Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 106.662871] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.696Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 106.666059] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.699Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 106.669866] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.703Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 106.672487] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.706Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 106.675160] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.708Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 106.678164] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.711Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 106.680876] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.714Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 106.686779] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.720Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 106.691861] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.725Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 106.694880] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.728Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 106.729668] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.763Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 106.732529] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.766Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 106.735352] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.769Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 106.738033] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.771Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 106.741434] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.775Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 106.744378] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.778Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 106.747238] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.781Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 106.750367] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.784Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 106.752874] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.786Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 106.755809] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.789Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 106.758439] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.792Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 106.761433] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.795Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 106.763986] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.797Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 106.766913] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.800Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 106.769439] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.803Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 106.772218] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.806Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 106.775102] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.808Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 106.778543] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.812Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 106.780953] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.814Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 106.783548] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.817Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 106.785907] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.819Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 106.788713] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.822Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 106.790833] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.824Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 106.792867] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.826Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 106.795129] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.828Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 106.796567] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.830Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 106.798810] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.832Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 106.800718] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.834Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 106.803137] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.836Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 106.805543] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.839Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 106.808281] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.842Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 106.810241] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.844Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 106.812578] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.846Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 106.821735] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.855Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 106.823869] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.857Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 106.825787] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.859Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 106.828984] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.862Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 106.831039] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.864Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 106.833799] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.867Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 106.836155] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.870Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 106.838372] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.872Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 106.840316] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.874Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 106.843225] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.877Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 106.845148] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:14:55.879Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 108.494092] gitlab-workhorse[824]: time="2026-09-06T16:14:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.503855] gitlab-workhorse[824]: time="2026-09-06T16:15:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.504514] gitlab-workhorse[824]: time="2026-09-06T16:15:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 130.388487] postgres[2087]: [2087] ERROR: relation "table_without_models" does not exist gitlab # [ 130.388898] postgres[2087]: [2087] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 130.907664] postgres[2107]: [2107] ERROR: relation "table_without_models" does not exist gitlab # [ 130.909266] postgres[2107]: [2107] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 131.280255] sidekiq-cluster[1640]: InitializerConnections Query: SET client_min_messages TO 'warning' gitlab # [ 131.281360] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.282498] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.283230] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.283966] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.284660] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.285529] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.286272] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.304863] sidekiq-cluster[1640]: InitializerConnections Query: SET standard_conforming_strings = on gitlab # [ 131.306105] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.307226] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.307876] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.308717] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.309578] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.310268] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.310927] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.340202] sidekiq-cluster[1640]: InitializerConnections Query: SET intervalstyle = iso_8601 gitlab # [ 131.343504] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.344590] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.345264] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.345963] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.346702] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.347614] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.348286] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.355311] sidekiq-cluster[1640]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname gitlab # [ 131.356726] sidekiq-cluster[1640]: FROM pg_type as t gitlab # [ 131.357637] sidekiq-cluster[1640]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz') gitlab # [ 131.358512] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.359176] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.359861] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.360785] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.361619] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.362286] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.362947] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.375384] sidekiq-cluster[1640]: InitializerConnections Query: SET SESSION timezone TO 'UTC' gitlab # [ 131.376670] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.378053] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.379546] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.380567] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.381230] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.381866] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.382716] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.433505] sidekiq-cluster[1640]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'sidekiq_throttling_middleware' gitlab # [ 131.434752] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.435688] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.436557] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.437247] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.437906] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.438776] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.439613] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 132.896820] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:21.929Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 132.901815] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:21.935Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 132.903215] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:21.935Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 132.904384] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:21.935Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 132.905627] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:21.937Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 134.315473] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:23.348Z","message":"Cleaning working queues"} gitlab # [ 134.320414] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:23.353Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c0cb240082bd1c53f3b3525a","created_at":"2026-09-06T16:14:24.604Z","trace_propagation_headers":{"sentry-trace":"66279b25b5524d5d9cf6f4af3f8d10b7-6cd7f5037cce4473","baggage":"sentry-trace_id=66279b25b5524d5d9cf6f4af3f8d10b7,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"60bfd58f6ed4db12301f2e7b13a3574b","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:14:24.611Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c0cb240082bd1c53f3b3525a: start","job_status":"start","queue_duration_s":58.742135,"scheduling_latency_s":58.742207} gitlab # [ 134.523873] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:23.557Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c0cb240082bd1c53f3b3525a","created_at":"2026-09-06T16:14:24.604Z","trace_propagation_headers":{"sentry-trace":"66279b25b5524d5d9cf6f4af3f8d10b7-6cd7f5037cce4473","baggage":"sentry-trace_id=66279b25b5524d5d9cf6f4af3f8d10b7,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"60bfd58f6ed4db12301f2e7b13a3574b","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:14:24.611Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c0cb240082bd1c53f3b3525a: done: 0.203725 sec","job_status":"done","queue_duration_s":58.742135,"scheduling_latency_s":58.742207,"redis_calls":29,"redis_duration_s":0.007305000000000001,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004178,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001892,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001235,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08360252500000001,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.203725,"completed_at":"2026-09-06T16:15:23.557Z","load_balancing_strategy":"primary","db_duration_s":0.039118,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 134.920661] bundle[1610]: {"timestamp":"2026-09-06T16:15:23.953Z","pid":1610,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 134.922314] bundle[1610]: {"timestamp":"2026-09-06T16:15:23.953Z","pid":1610,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 134.923414] bundle[1610]: {"timestamp":"2026-09-06T16:15:23.953Z","pid":1610,"message":"! #\u003cThread:0x00007e38e248a9d8 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 134.929318] bundle[1610]: {"timestamp":"2026-09-06T16:15:23.953Z","pid":1610,"message":"! #\u003cThread:0x00007e38e266eda8 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 134.930923] bundle[1610]: {"timestamp":"2026-09-06T16:15:23.953Z","pid":1610,"message":"Use Ctrl-C to stop"} gitlab # [ 134.932478] bundle[1610]: {"timestamp":"2026-09-06T16:15:23.954Z","pid":1610,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 135.661496] bundle[1610]: {"timestamp":"2026-09-06T16:15:24.695Z","pid":1610,"message":"- Worker 0 (PID: 2190) booted in 0.7s, phase: 0"} gitlab # [ 135.730763] bundle[1610]: {"timestamp":"2026-09-06T16:15:24.764Z","pid":1610,"message":"- Worker 1 (PID: 2194) booted in 0.75s, phase: 0"} gitlab # [ 135.735265] systemd[1]: Started gitlab.service. gitlab # [ 135.737097] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 135.739589] systemd[1]: Starting setup-gitlab.service... gitlab # [ 135.762797] setup-gitlab-start[2227]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 137.669165] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:15:26 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 1885 gitlab # [ 137.677291] setup-gitlab-start[2227]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 137.678278] setup-gitlab-start[2227]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 137.694785] runuser[2244]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 138.061764] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.067125] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.071920] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.641641] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.646586] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.651305] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 144.480888] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.513Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"78a91c48892053504f53b6cc","created_at":"2026-09-06T16:15:33.477Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a9f2ee932f8470e3b2ac61a6f67131a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.510Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-78a91c48892053504f53b6cc: start","job_status":"start","queue_duration_s":0.002779,"scheduling_latency_s":0.00282} gitlab # [ 144.600444] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.633Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"78a91c48892053504f53b6cc","created_at":"2026-09-06T16:15:33.477Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a9f2ee932f8470e3b2ac61a6f67131a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.510Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-78a91c48892053504f53b6cc: done: 0.120154 sec","job_status":"done","queue_duration_s":0.002779,"scheduling_latency_s":0.00282,"redis_calls":16,"redis_duration_s":0.021621,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.006317,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002603,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00086212,"duration_s":0.120154,"completed_at":"2026-09-06T16:15:33.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.650098] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.683Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f232ca93b94194c1a3dcddfb","created_at":"2026-09-06T16:15:33.541Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f9acfd77b0424af0cc3e7b46d586d73","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:15:33.556Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f232ca93b94194c1a3dcddfb: start","job_status":"start","queue_duration_s":0.126902,"scheduling_latency_s":0.126935} gitlab # [ 144.698481] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.732Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 144.741630] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.774Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f232ca93b94194c1a3dcddfb","created_at":"2026-09-06T16:15:33.541Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f9acfd77b0424af0cc3e7b46d586d73","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:15:33.556Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f232ca93b94194c1a3dcddfb: done: 0.091681 sec","job_status":"done","queue_duration_s":0.126902,"scheduling_latency_s":0.126935,"redis_calls":10,"redis_duration_s":0.034475,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000903,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.030068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003504,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.024187153,"duration_s":0.091681,"completed_at":"2026-09-06T16:15:33.774Z","load_balancing_strategy":"primary","db_duration_s":0.001017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.781391] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.784Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a1b2aa00fd142914a633e236","created_at":"2026-09-06T16:15:33.568Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.606Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a1b2aa00fd142914a633e236: start","job_status":"start","queue_duration_s":0.177321,"scheduling_latency_s":0.177441} gitlab # [ 144.852618] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.885Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a1b2aa00fd142914a633e236","created_at":"2026-09-06T16:15:33.568Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.606Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a1b2aa00fd142914a633e236: done: 0.101783 sec","job_status":"done","queue_duration_s":0.177321,"scheduling_latency_s":0.177441,"redis_calls":19,"redis_duration_s":0.021638,"redis_read_bytes":21,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004556,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.001389,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.010568,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.005125,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001511924,"duration_s":0.101783,"completed_at":"2026-09-06T16:15:33.885Z","load_balancing_strategy":"primary","db_duration_s":0.012005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.899148] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"56f5b4e2b8417b64c95a8e27","created_at":"2026-09-06T16:15:33.628Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62721819126b8c940789f579c06fe828","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.681Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-56f5b4e2b8417b64c95a8e27: start","job_status":"start","queue_duration_s":0.250889,"scheduling_latency_s":0.250927} gitlab # [ 144.958836] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.992Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"56f5b4e2b8417b64c95a8e27","created_at":"2026-09-06T16:15:33.628Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62721819126b8c940789f579c06fe828","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.681Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-56f5b4e2b8417b64c95a8e27: done: 0.059936 sec","job_status":"done","queue_duration_s":0.250889,"scheduling_latency_s":0.250927,"redis_calls":10,"redis_duration_s":0.010672000000000001,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00174,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006935,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003191467,"duration_s":0.059936,"completed_at":"2026-09-06T16:15:33.992Z","load_balancing_strategy":"primary","db_duration_s":0.017408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.980928] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:33.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3d2b6fe25140d5fb0c4c07ab","created_at":"2026-09-06T16:15:33.706Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.724Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3d2b6fe25140d5fb0c4c07ab: start","job_status":"start","queue_duration_s":0.268757,"scheduling_latency_s":0.268774} gitlab # [ 145.208747] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.242Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3d2b6fe25140d5fb0c4c07ab","created_at":"2026-09-06T16:15:33.706Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.724Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3d2b6fe25140d5fb0c4c07ab: done: 0.24835 sec","job_status":"done","queue_duration_s":0.268757,"scheduling_latency_s":0.268774,"redis_calls":71,"redis_duration_s":0.054230999999999994,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.008716,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001253,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.042439,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001823,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.105121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001228927,"duration_s":0.24835,"completed_at":"2026-09-06T16:15:34.241Z","load_balancing_strategy":"primary","db_duration_s":0.056485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.273759] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"fee0007780de9f8f531a0897","created_at":"2026-09-06T16:15:33.763Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"072b41fd4f95a381e1500641c9b999ed","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.781Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-fee0007780de9f8f531a0897: start","job_status":"start","queue_duration_s":0.463768,"scheduling_latency_s":0.463787} gitlab # [ 145.286621] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.267Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"fee0007780de9f8f531a0897","created_at":"2026-09-06T16:15:33.763Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"072b41fd4f95a381e1500641c9b999ed","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.781Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-fee0007780de9f8f531a0897: done: 0.022284 sec","job_status":"done","queue_duration_s":0.463768,"scheduling_latency_s":0.463787,"redis_calls":11,"redis_duration_s":0.001729,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000304,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000234,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000316,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222375,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.022284,"completed_at":"2026-09-06T16:15:34.267Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.327423] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a784b634981fbe59084d860d","created_at":"2026-09-06T16:15:33.797Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5aab43490b2c0d81df0c9d74592fc5f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.819Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a784b634981fbe59084d860d: start","job_status":"start","queue_duration_s":0.449189,"scheduling_latency_s":0.449203} gitlab # [ 145.341071] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.276Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 145.344596] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a784b634981fbe59084d860d","created_at":"2026-09-06T16:15:33.797Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5aab43490b2c0d81df0c9d74592fc5f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.819Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a784b634981fbe59084d860d: done: 0.013034 sec","job_status":"done","queue_duration_s":0.449189,"scheduling_latency_s":0.449203,"redis_calls":10,"redis_duration_s":0.001392,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000298,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222095,"duration_s":0.013034,"completed_at":"2026-09-06T16:15:34.281Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.345810] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.282Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"7697ba315eb450973a68d4bf","created_at":"2026-09-06T16:15:33.830Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a3bcf75665731dc8bb665a6064df561","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.843Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-7697ba315eb450973a68d4bf: start","job_status":"start","queue_duration_s":0.439363,"scheduling_latency_s":0.439378} gitlab # [ 145.386869] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"7697ba315eb450973a68d4bf","created_at":"2026-09-06T16:15:33.830Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a3bcf75665731dc8bb665a6064df561","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.843Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-7697ba315eb450973a68d4bf: done: 0.015705 sec","job_status":"done","queue_duration_s":0.439363,"scheduling_latency_s":0.439378,"redis_calls":12,"redis_duration_s":0.00175,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00065,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00034753,"exclusive_lock_hold_duration_s":0.002725206,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.015705,"completed_at":"2026-09-06T16:15:34.298Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.393681] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.299Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"04124fe8942fc7423ff2e710","created_at":"2026-09-06T16:15:33.822Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c69f5727f731427e","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.851Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-04124fe8942fc7423ff2e710: start","job_status":"start","queue_duration_s":0.447842,"scheduling_latency_s":0.447855} gitlab # [ 145.405691] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.316Z","message":"No existing merge request to be cleaned up."} gitlab # [ 145.407075] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.325Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"04124fe8942fc7423ff2e710","created_at":"2026-09-06T16:15:33.822Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c69f5727f731427e","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.851Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-04124fe8942fc7423ff2e710: done: 0.026213 sec","job_status":"done","queue_duration_s":0.447842,"scheduling_latency_s":0.447855,"redis_calls":13,"redis_duration_s":0.002001,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000267631,"duration_s":0.026213,"completed_at":"2026-09-06T16:15:34.325Z","load_balancing_strategy":"primary","db_duration_s":0.003062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.464966] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.326Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fbe387e4bf857eb0a9ebdd46","created_at":"2026-09-06T16:15:33.822Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c69f5727f731427e","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.851Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fbe387e4bf857eb0a9ebdd46: start","job_status":"start","queue_duration_s":0.475002,"scheduling_latency_s":0.475015} gitlab # [ 145.465780] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.333Z","message":"No existing merge request to be cleaned up."} gitlab # [ 145.466749] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.338Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fbe387e4bf857eb0a9ebdd46","created_at":"2026-09-06T16:15:33.822Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c69f5727f731427e","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.851Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fbe387e4bf857eb0a9ebdd46: done: 0.012375 sec","job_status":"done","queue_duration_s":0.475002,"scheduling_latency_s":0.475015,"redis_calls":10,"redis_duration_s":0.001523,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012375,"completed_at":"2026-09-06T16:15:34.338Z","load_balancing_strategy":"primary","db_duration_s":0.001275,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.492117] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.340Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7feafe475491b3021f861819","created_at":"2026-09-06T16:15:33.822Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c69f5727f731427e","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.851Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7feafe475491b3021f861819: start","job_status":"start","queue_duration_s":0.488483,"scheduling_latency_s":0.488496} gitlab # [ 145.502769] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.347Z","message":"No existing merge request to be cleaned up."} gitlab # [ 145.504242] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.351Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7feafe475491b3021f861819","created_at":"2026-09-06T16:15:33.822Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c69f5727f731427e","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.851Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7feafe475491b3021f861819: done: 0.011697 sec","job_status":"done","queue_duration_s":0.488483,"scheduling_latency_s":0.488496,"redis_calls":10,"redis_duration_s":0.001469,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000865,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011697,"completed_at":"2026-09-06T16:15:34.351Z","load_balancing_strategy":"primary","db_duration_s":0.001071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.547849] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.352Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c73d46c2ff82e32dd88b8d39","created_at":"2026-09-06T16:15:33.822Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c69f5727f731427e","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.851Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c73d46c2ff82e32dd88b8d39: start","job_status":"start","queue_duration_s":0.501401,"scheduling_latency_s":0.501422} gitlab # [ 145.548591] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.359Z","message":"No existing merge request to be cleaned up."} gitlab # [ 145.550097] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.364Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c73d46c2ff82e32dd88b8d39","created_at":"2026-09-06T16:15:33.822Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c69f5727f731427e","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"793954f605a45c385c0af1623ce35444","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.851Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c73d46c2ff82e32dd88b8d39: done: 0.011941 sec","job_status":"done","queue_duration_s":0.501401,"scheduling_latency_s":0.501422,"redis_calls":10,"redis_duration_s":0.001494,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011941,"completed_at":"2026-09-06T16:15:34.364Z","load_balancing_strategy":"primary","db_duration_s":0.001091,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.585135] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","jid":"0a5d4b1f9522e30226700e32","created_at":"2026-09-06T16:15:33.851Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3b8e84ed65bc7d9c0f12b3aea496ecd","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.861Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::StuckProjectImportJobsWorker JID-0a5d4b1f9522e30226700e32: start","job_status":"start","queue_duration_s":0.504661,"scheduling_latency_s":0.504675} gitlab # [ 145.655934] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.689Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","jid":"0a5d4b1f9522e30226700e32","created_at":"2026-09-06T16:15:33.851Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3b8e84ed65bc7d9c0f12b3aea496ecd","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.861Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::StuckProjectImportJobsWorker JID-0a5d4b1f9522e30226700e32: done: 0.323258 sec","job_status":"done","queue_duration_s":0.504661,"scheduling_latency_s":0.504675,"redis_calls":10,"redis_duration_s":0.0023090000000000003,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000545,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001342,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.294,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.294,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000905423,"duration_s":0.323258,"completed_at":"2026-09-06T16:15:34.689Z","load_balancing_strategy":"primary","db_duration_s":0.294601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.695185] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"a953899b72e2dfb954586d09","created_at":"2026-09-06T16:15:33.883Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34fbc34a675903f1086413ccb79e9878","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.949Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-a953899b72e2dfb954586d09: start","job_status":"start","queue_duration_s":0.778424,"scheduling_latency_s":0.778469} gitlab # [ 145.745612] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"a953899b72e2dfb954586d09","created_at":"2026-09-06T16:15:33.883Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34fbc34a675903f1086413ccb79e9878","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.949Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-a953899b72e2dfb954586d09: done: 0.050106 sec","job_status":"done","queue_duration_s":0.778424,"scheduling_latency_s":0.778469,"redis_calls":10,"redis_duration_s":0.002477,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000471,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001509,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000490286,"duration_s":0.050106,"completed_at":"2026-09-06T16:15:34.778Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.785699] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.819Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMigrateExternalDiffsWorker","jid":"affc9f29812c55e6c1687094","created_at":"2026-09-06T16:15:33.964Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"037f36d31f03bb738799a83ebadee3cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.975Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMigrateExternalDiffsWorker JID-affc9f29812c55e6c1687094: start","job_status":"start","queue_duration_s":0.843603,"scheduling_latency_s":0.843648} gitlab # [ 145.830936] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.864Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMigrateExternalDiffsWorker","jid":"affc9f29812c55e6c1687094","created_at":"2026-09-06T16:15:33.964Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"037f36d31f03bb738799a83ebadee3cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.975Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMigrateExternalDiffsWorker JID-affc9f29812c55e6c1687094: done: 0.045223 sec","job_status":"done","queue_duration_s":0.843603,"scheduling_latency_s":0.843648,"redis_calls":12,"redis_duration_s":0.003083,"redis_read_bytes":11,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":311,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":533,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000881397,"exclusive_lock_hold_duration_s":0.000700369,"duration_s":0.045223,"completed_at":"2026-09-06T16:15:34.864Z","load_balancing_strategy":"primary","db_duration_s":0.001435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.854600] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7c57821edded326358364b57","created_at":"2026-09-06T16:15:33.985Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e15510e3b050d56bae8ca04efa963ec8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.996Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7c57821edded326358364b57: start","job_status":"start","queue_duration_s":0.870312,"scheduling_latency_s":0.870347} gitlab # [ 145.922683] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:34.955Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7c57821edded326358364b57","created_at":"2026-09-06T16:15:33.985Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e15510e3b050d56bae8ca04efa963ec8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:33.996Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7c57821edded326358364b57: done: 0.088577 sec","job_status":"done","queue_duration_s":0.870312,"scheduling_latency_s":0.870347,"redis_calls":12,"redis_duration_s":0.004325,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000697,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001931,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001697,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000990908,"exclusive_lock_hold_duration_s":0.039389084000000005,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.088577,"completed_at":"2026-09-06T16:15:34.955Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.967526] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5f78b47289be403ed55e4721","created_at":"2026-09-06T16:15:34.014Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d51eb8dec40e223b3e43c6b7dda6778","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:15:34.017Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5f78b47289be403ed55e4721: start","job_status":"start","queue_duration_s":0.983206,"scheduling_latency_s":0.983245} gitlab # [ 146.000618] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5f78b47289be403ed55e4721","created_at":"2026-09-06T16:15:34.014Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d51eb8dec40e223b3e43c6b7dda6778","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:15:34.017Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5f78b47289be403ed55e4721: done: 0.033263 sec","job_status":"done","queue_duration_s":0.983206,"scheduling_latency_s":0.983245,"redis_calls":10,"redis_duration_s":0.002775,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000562,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000304228,"duration_s":0.033263,"completed_at":"2026-09-06T16:15:35.034Z","load_balancing_strategy":"primary","db_duration_s":0.004869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.038855] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"248cbc13b875c01be488f7a7","created_at":"2026-09-06T16:15:34.021Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c06f2da847322b4c59237b89225c740","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.024Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-248cbc13b875c01be488f7a7: start","job_status":"start","queue_duration_s":1.04795,"scheduling_latency_s":1.047975} gitlab # [ 146.066625] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"248cbc13b875c01be488f7a7","created_at":"2026-09-06T16:15:34.021Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c06f2da847322b4c59237b89225c740","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.024Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-248cbc13b875c01be488f7a7: done: 0.027733 sec","job_status":"done","queue_duration_s":1.04795,"scheduling_latency_s":1.047975,"redis_calls":13,"redis_duration_s":0.0033439999999999998,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000544,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001607,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009807670000000001,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.027733,"completed_at":"2026-09-06T16:15:35.100Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003569,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 146.107671] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.141Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0ffcb46684e1f0d28eb10c2e","created_at":"2026-09-06T16:15:34.028Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b20c6aaa5716469c6157d321b9476b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.030Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0ffcb46684e1f0d28eb10c2e: start","job_status":"start","queue_duration_s":1.110109,"scheduling_latency_s":1.110135} gitlab # [ 146.182922] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0ffcb46684e1f0d28eb10c2e","created_at":"2026-09-06T16:15:34.028Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b20c6aaa5716469c6157d321b9476b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.030Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0ffcb46684e1f0d28eb10c2e: done: 0.074894 sec","job_status":"done","queue_duration_s":1.110109,"scheduling_latency_s":1.110135,"redis_calls":14,"redis_duration_s":0.00349,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000976,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001391,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001123,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00107807,"exclusive_lock_hold_duration_s":0.031907964999999996,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.074894,"completed_at":"2026-09-06T16:15:35.215Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.232864] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"79539dd749a836777d48ed7a","created_at":"2026-09-06T16:15:34.033Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9dd13d0517525d4ee6a3decb9271b2d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.036Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-79539dd749a836777d48ed7a: start","job_status":"start","queue_duration_s":1.229217,"scheduling_latency_s":1.229258} gitlab # [ 146.263438] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.296Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 146.276767] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"79539dd749a836777d48ed7a","created_at":"2026-09-06T16:15:34.033Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9dd13d0517525d4ee6a3decb9271b2d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.036Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-79539dd749a836777d48ed7a: done: 0.043605 sec","job_status":"done","queue_duration_s":1.229217,"scheduling_latency_s":1.229258,"redis_calls":10,"redis_duration_s":0.0030499999999999998,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000454,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001929,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000667,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000548952,"duration_s":0.043605,"completed_at":"2026-09-06T16:15:35.309Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.316584] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"fb35fa99a73106c9892bb5d6","created_at":"2026-09-06T16:15:34.054Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c832727da91ae722d7447ab28013f16b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.066Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-fb35fa99a73106c9892bb5d6: start","job_status":"start","queue_duration_s":1.283605,"scheduling_latency_s":1.283646} gitlab # [ 146.363227] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"fb35fa99a73106c9892bb5d6","created_at":"2026-09-06T16:15:34.054Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c832727da91ae722d7447ab28013f16b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.066Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-fb35fa99a73106c9892bb5d6: done: 0.046672 sec","job_status":"done","queue_duration_s":1.283605,"scheduling_latency_s":1.283646,"redis_calls":12,"redis_duration_s":0.002746,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000907,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000430223,"duration_s":0.046672,"completed_at":"2026-09-06T16:15:35.396Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.402793] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.436Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"38e6adf5949d91cb2e4c956c","created_at":"2026-09-06T16:15:34.077Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"609d5f5d2e6a1354227f9d19fdd19bcf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.086Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-38e6adf5949d91cb2e4c956c: start","job_status":"start","queue_duration_s":1.34964,"scheduling_latency_s":1.349683} gitlab # [ 146.446392] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.479Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"38e6adf5949d91cb2e4c956c","created_at":"2026-09-06T16:15:34.077Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"609d5f5d2e6a1354227f9d19fdd19bcf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.086Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-38e6adf5949d91cb2e4c956c: done: 0.043463 sec","job_status":"done","queue_duration_s":1.34964,"scheduling_latency_s":1.349683,"redis_calls":10,"redis_duration_s":0.002693,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000783,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001362,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000548,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000488609,"duration_s":0.043463,"completed_at":"2026-09-06T16:15:35.479Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.486205] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"04f6a51a4e5f84ed3979cff7","created_at":"2026-09-06T16:15:34.102Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"335fa4e3db2d5ec2e5cc4a7a09b0dc1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.110Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-04f6a51a4e5f84ed3979cff7: start","job_status":"start","queue_duration_s":1.40867,"scheduling_latency_s":1.408713} gitlab # [ 146.519900] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.552Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 146.535274] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"04f6a51a4e5f84ed3979cff7","created_at":"2026-09-06T16:15:34.102Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"335fa4e3db2d5ec2e5cc4a7a09b0dc1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.110Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-04f6a51a4e5f84ed3979cff7: done: 0.048655 sec","job_status":"done","queue_duration_s":1.40867,"scheduling_latency_s":1.408713,"redis_calls":10,"redis_duration_s":0.006737000000000001,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000440558,"duration_s":0.048655,"completed_at":"2026-09-06T16:15:35.568Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.575425] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a8b7054dd9f030e4d8694b1a","created_at":"2026-09-06T16:15:34.119Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e583098335ea1e33614597a9874534f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.128Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a8b7054dd9f030e4d8694b1a: start","job_status":"start","queue_duration_s":1.480526,"scheduling_latency_s":1.480568} gitlab # [ 146.950388] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:35.983Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a8b7054dd9f030e4d8694b1a","created_at":"2026-09-06T16:15:34.119Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e583098335ea1e33614597a9874534f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.128Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a8b7054dd9f030e4d8694b1a: done: 0.374945 sec","job_status":"done","queue_duration_s":1.480526,"scheduling_latency_s":1.480568,"redis_calls":565,"redis_duration_s":0.110152,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000669,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002023,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.10746,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.27981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000527442,"duration_s":0.374945,"completed_at":"2026-09-06T16:15:35.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.991137] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c3956becf39f936347dfc773","created_at":"2026-09-06T16:15:34.136Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a871e3a6cabd98ad70b483c9bb60309","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.144Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c3956becf39f936347dfc773: start","job_status":"start","queue_duration_s":1.880009,"scheduling_latency_s":1.880048} gitlab # [ 147.193052] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c3956becf39f936347dfc773","created_at":"2026-09-06T16:15:34.136Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a871e3a6cabd98ad70b483c9bb60309","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.144Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c3956becf39f936347dfc773: done: 0.201277 sec","job_status":"done","queue_duration_s":1.880009,"scheduling_latency_s":1.880048,"redis_calls":12,"redis_duration_s":0.0035329999999999997,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000498,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001743,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001292,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.037,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000804292,"exclusive_lock_hold_duration_s":0.16044972200000002,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.201277,"completed_at":"2026-09-06T16:15:36.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057465,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.195558] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.228Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"67966a69b551e6e6545ebaeb","created_at":"2026-09-06T16:15:34.155Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"042359a3dec7235e47a529fc95f0ca7d","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.162Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-67966a69b551e6e6545ebaeb: start","job_status":"start","queue_duration_s":2.066152,"scheduling_latency_s":2.066186} gitlab # [ 147.264151] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"67966a69b551e6e6545ebaeb","created_at":"2026-09-06T16:15:34.155Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"042359a3dec7235e47a529fc95f0ca7d","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.162Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-67966a69b551e6e6545ebaeb: done: 0.028728 sec","job_status":"done","queue_duration_s":2.066152,"scheduling_latency_s":2.066186,"redis_calls":11,"redis_duration_s":0.002817,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001649,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341105,"duration_s":0.028728,"completed_at":"2026-09-06T16:15:36.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.306848] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.259Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"d423c98ec8fe46345945bab2","created_at":"2026-09-06T16:15:34.171Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ba5ca9872b27d316ba6424506a177d9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.179Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-d423c98ec8fe46345945bab2: start","job_status":"start","queue_duration_s":2.07936,"scheduling_latency_s":2.079375} gitlab # [ 147.307660] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.276Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"d423c98ec8fe46345945bab2","created_at":"2026-09-06T16:15:34.171Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ba5ca9872b27d316ba6424506a177d9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.179Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-d423c98ec8fe46345945bab2: done: 0.017711 sec","job_status":"done","queue_duration_s":2.07936,"scheduling_latency_s":2.079375,"redis_calls":11,"redis_duration_s":0.001519,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213993,"duration_s":0.017711,"completed_at":"2026-09-06T16:15:36.276Z","load_balancing_strategy":"primary","db_duration_s":0.002076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.361915] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.277Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f3f5dc4812df53de788c6ebc","created_at":"2026-09-06T16:15:34.186Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af8d105e6110645cc8f050171aac1702","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.192Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f3f5dc4812df53de788c6ebc: start","job_status":"start","queue_duration_s":2.08521,"scheduling_latency_s":2.085224} gitlab # [ 147.362733] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f3f5dc4812df53de788c6ebc","created_at":"2026-09-06T16:15:34.186Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af8d105e6110645cc8f050171aac1702","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.192Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f3f5dc4812df53de788c6ebc: done: 0.030696 sec","job_status":"done","queue_duration_s":2.08521,"scheduling_latency_s":2.085224,"redis_calls":10,"redis_duration_s":0.001457,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000312,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000240813,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.030696,"completed_at":"2026-09-06T16:15:36.308Z","load_balancing_strategy":"primary","db_duration_s":0.015575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.379156] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0bd626d457bf0beb2ad59cb6","created_at":"2026-09-06T16:15:34.202Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bec63ab0a494d5e9d27e9bd73fac32f5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.207Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0bd626d457bf0beb2ad59cb6: start","job_status":"start","queue_duration_s":2.102367,"scheduling_latency_s":2.102383} gitlab # [ 147.426920] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.327Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0bd626d457bf0beb2ad59cb6","created_at":"2026-09-06T16:15:34.202Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bec63ab0a494d5e9d27e9bd73fac32f5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.207Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0bd626d457bf0beb2ad59cb6: done: 0.018158 sec","job_status":"done","queue_duration_s":2.102367,"scheduling_latency_s":2.102383,"redis_calls":12,"redis_duration_s":0.001789,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000315,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000679,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000350883,"exclusive_lock_hold_duration_s":0.004949233,"duration_s":0.018158,"completed_at":"2026-09-06T16:15:36.327Z","load_balancing_strategy":"primary","db_duration_s":0.002935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.468299] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.329Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"b2cc24c3ca0e9f2478421acd","created_at":"2026-09-06T16:15:34.214Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a95bbdf1fe443a02172ebda3908802fc","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.220Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-b2cc24c3ca0e9f2478421acd: start","job_status":"start","queue_duration_s":2.108705,"scheduling_latency_s":2.108719} gitlab # [ 147.469039] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.352Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"b2cc24c3ca0e9f2478421acd","created_at":"2026-09-06T16:15:34.214Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a95bbdf1fe443a02172ebda3908802fc","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.220Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-b2cc24c3ca0e9f2478421acd: done: 0.02351 sec","job_status":"done","queue_duration_s":2.108705,"scheduling_latency_s":2.108719,"redis_calls":10,"redis_duration_s":0.0018750000000000001,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001242,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000369,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000356749,"duration_s":0.02351,"completed_at":"2026-09-06T16:15:36.352Z","load_balancing_strategy":"primary","db_duration_s":0.003852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.517299] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4775222b8df96f876add20a8","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4775222b8df96f876add20a8: start","job_status":"start","queue_duration_s":2.122963,"scheduling_latency_s":2.122978} gitlab # [ 147.518270] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4775222b8df96f876add20a8","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4775222b8df96f876add20a8: done: 0.016388 sec","job_status":"done","queue_duration_s":2.122963,"scheduling_latency_s":2.122978,"redis_calls":16,"redis_duration_s":0.002638,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226286,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016388,"completed_at":"2026-09-06T16:15:36.370Z","load_balancing_strategy":"primary","db_duration_s":0.000362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.570918] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64d7b81b8e3b3c2fe0c8fc71","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64d7b81b8e3b3c2fe0c8fc71: start","job_status":"start","queue_duration_s":2.140409,"scheduling_latency_s":2.140422} gitlab # [ 147.571842] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64d7b81b8e3b3c2fe0c8fc71","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64d7b81b8e3b3c2fe0c8fc71: done: 0.009913 sec","job_status":"done","queue_duration_s":2.140409,"scheduling_latency_s":2.140422,"redis_calls":13,"redis_duration_s":0.001745,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009913,"completed_at":"2026-09-06T16:15:36.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.580086] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d76c45a355b1e9e9d06deac6","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d76c45a355b1e9e9d06deac6: start","job_status":"start","queue_duration_s":2.151341,"scheduling_latency_s":2.151355} gitlab # [ 147.593135] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d76c45a355b1e9e9d06deac6","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d76c45a355b1e9e9d06deac6: done: 0.009643 sec","job_status":"done","queue_duration_s":2.151341,"scheduling_latency_s":2.151355,"redis_calls":13,"redis_duration_s":0.001672,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009643,"completed_at":"2026-09-06T16:15:36.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.655831] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0df2a255184544250f726ba4","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0df2a255184544250f726ba4: start","job_status":"start","queue_duration_s":2.162005,"scheduling_latency_s":2.162043} gitlab # [ 147.656886] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0df2a255184544250f726ba4","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0df2a255184544250f726ba4: done: 0.011132 sec","job_status":"done","queue_duration_s":2.162005,"scheduling_latency_s":2.162043,"redis_calls":13,"redis_duration_s":0.002922,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011132,"completed_at":"2026-09-06T16:15:36.404Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.709067] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"320815b31a44909b668c4361","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-320815b31a44909b668c4361: start","job_status":"start","queue_duration_s":2.174251,"scheduling_latency_s":2.174263} gitlab # [ 147.709846] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"320815b31a44909b668c4361","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-320815b31a44909b668c4361: done: 0.010766 sec","job_status":"done","queue_duration_s":2.174251,"scheduling_latency_s":2.174263,"redis_calls":13,"redis_duration_s":0.002618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010766,"completed_at":"2026-09-06T16:15:36.415Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.753852] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c501550a0616cad2e732e2e7","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c501550a0616cad2e732e2e7: start","job_status":"start","queue_duration_s":2.186092,"scheduling_latency_s":2.186106} gitlab # [ 147.754681] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c501550a0616cad2e732e2e7","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c501550a0616cad2e732e2e7: done: 0.01075 sec","job_status":"done","queue_duration_s":2.186092,"scheduling_latency_s":2.186106,"redis_calls":13,"redis_duration_s":0.002572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001792,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01075,"completed_at":"2026-09-06T16:15:36.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.800795] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af1568fb029d52ad83a96a39","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af1568fb029d52ad83a96a39: start","job_status":"start","queue_duration_s":2.198299,"scheduling_latency_s":2.198314} gitlab # [ 147.801688] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af1568fb029d52ad83a96a39","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af1568fb029d52ad83a96a39: done: 0.011002 sec","job_status":"done","queue_duration_s":2.198299,"scheduling_latency_s":2.198314,"redis_calls":13,"redis_duration_s":0.002593,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001822,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011002,"completed_at":"2026-09-06T16:15:36.440Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.850735] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7585a38fceee6b34c9dd294","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7585a38fceee6b34c9dd294: start","job_status":"start","queue_duration_s":2.21035,"scheduling_latency_s":2.210362} gitlab # [ 147.851760] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7585a38fceee6b34c9dd294","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7585a38fceee6b34c9dd294: done: 0.010759 sec","job_status":"done","queue_duration_s":2.21035,"scheduling_latency_s":2.210362,"redis_calls":13,"redis_duration_s":0.002689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000772,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010759,"completed_at":"2026-09-06T16:15:36.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.900660] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ab3ced0f3d2cddc581f8800","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ab3ced0f3d2cddc581f8800: start","job_status":"start","queue_duration_s":2.222161,"scheduling_latency_s":2.222175} gitlab # [ 147.901586] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ab3ced0f3d2cddc581f8800","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ab3ced0f3d2cddc581f8800: done: 0.010702 sec","job_status":"done","queue_duration_s":2.222161,"scheduling_latency_s":2.222175,"redis_calls":13,"redis_duration_s":0.002609,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010702,"completed_at":"2026-09-06T16:15:36.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.950851] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3bad2ec246cf2fda27961f4","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3bad2ec246cf2fda27961f4: start","job_status":"start","queue_duration_s":2.234098,"scheduling_latency_s":2.234112} gitlab # [ 147.951851] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3bad2ec246cf2fda27961f4","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3bad2ec246cf2fda27961f4: done: 0.072831 sec","job_status":"done","queue_duration_s":2.234098,"scheduling_latency_s":2.234112,"redis_calls":13,"redis_duration_s":0.003502,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002306,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.072831,"completed_at":"2026-09-06T16:15:36.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.008823] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2795a238d9102c5d80043018","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2795a238d9102c5d80043018: start","job_status":"start","queue_duration_s":2.308385,"scheduling_latency_s":2.308408} gitlab # [ 148.009815] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2795a238d9102c5d80043018","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2795a238d9102c5d80043018: done: 0.011134 sec","job_status":"done","queue_duration_s":2.308385,"scheduling_latency_s":2.308408,"redis_calls":13,"redis_duration_s":0.0027530000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001825,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011134,"completed_at":"2026-09-06T16:15:36.550Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.052418] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21fa02e9a8f3051606fe3789","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21fa02e9a8f3051606fe3789: start","job_status":"start","queue_duration_s":2.3205,"scheduling_latency_s":2.320513} gitlab # [ 148.053246] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21fa02e9a8f3051606fe3789","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21fa02e9a8f3051606fe3789: done: 0.010199 sec","job_status":"done","queue_duration_s":2.3205,"scheduling_latency_s":2.320513,"redis_calls":13,"redis_duration_s":0.001809,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010199,"completed_at":"2026-09-06T16:15:36.561Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.097886] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9361daef649ffc91206b3d02","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9361daef649ffc91206b3d02: start","job_status":"start","queue_duration_s":2.331574,"scheduling_latency_s":2.331588} gitlab # [ 148.098680] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9361daef649ffc91206b3d02","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9361daef649ffc91206b3d02: done: 0.009786 sec","job_status":"done","queue_duration_s":2.331574,"scheduling_latency_s":2.331588,"redis_calls":13,"redis_duration_s":0.0016580000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009786,"completed_at":"2026-09-06T16:15:36.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.142117] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e78f1cefa1b4fe99f40077c2","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e78f1cefa1b4fe99f40077c2: start","job_status":"start","queue_duration_s":2.342233,"scheduling_latency_s":2.342246} gitlab # [ 148.142868] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e78f1cefa1b4fe99f40077c2","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e78f1cefa1b4fe99f40077c2: done: 0.009545 sec","job_status":"done","queue_duration_s":2.342233,"scheduling_latency_s":2.342246,"redis_calls":13,"redis_duration_s":0.0016220000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009545,"completed_at":"2026-09-06T16:15:36.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.161590] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"830bc779a62a259f443108f9","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-830bc779a62a259f443108f9: start","job_status":"start","queue_duration_s":2.353209,"scheduling_latency_s":2.353223} gitlab # [ 148.200073] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"830bc779a62a259f443108f9","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-830bc779a62a259f443108f9: done: 0.009518 sec","job_status":"done","queue_duration_s":2.353209,"scheduling_latency_s":2.353223,"redis_calls":13,"redis_duration_s":0.001586,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009518,"completed_at":"2026-09-06T16:15:36.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.206656] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37a0b680676854a6720a5da9","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37a0b680676854a6720a5da9: start","job_status":"start","queue_duration_s":2.3636,"scheduling_latency_s":2.363615} gitlab # [ 148.218094] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37a0b680676854a6720a5da9","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37a0b680676854a6720a5da9: done: 0.009558 sec","job_status":"done","queue_duration_s":2.3636,"scheduling_latency_s":2.363615,"redis_calls":13,"redis_duration_s":0.001619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009558,"completed_at":"2026-09-06T16:15:36.604Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.266807] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94b891614b07bb4a0e6c66b6","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94b891614b07bb4a0e6c66b6: start","job_status":"start","queue_duration_s":2.373987,"scheduling_latency_s":2.373999} gitlab # [ 148.267744] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94b891614b07bb4a0e6c66b6","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94b891614b07bb4a0e6c66b6: done: 0.009469 sec","job_status":"done","queue_duration_s":2.373987,"scheduling_latency_s":2.373999,"redis_calls":13,"redis_duration_s":0.001993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001424,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009469,"completed_at":"2026-09-06T16:15:36.614Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.300706] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3ee8048e64b18fb94c42329","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3ee8048e64b18fb94c42329: start","job_status":"start","queue_duration_s":2.384314,"scheduling_latency_s":2.384327} gitlab # [ 148.341765] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3ee8048e64b18fb94c42329","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3ee8048e64b18fb94c42329: done: 0.009487 sec","job_status":"done","queue_duration_s":2.384314,"scheduling_latency_s":2.384327,"redis_calls":13,"redis_duration_s":0.001635,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009487,"completed_at":"2026-09-06T16:15:36.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.347403] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f3dc212b1ea514c720e76fe","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f3dc212b1ea514c720e76fe: start","job_status":"start","queue_duration_s":2.395131,"scheduling_latency_s":2.395145} gitlab # [ 148.359222] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f3dc212b1ea514c720e76fe","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f3dc212b1ea514c720e76fe: done: 0.00972 sec","job_status":"done","queue_duration_s":2.395131,"scheduling_latency_s":2.395145,"redis_calls":13,"redis_duration_s":0.00183,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000708,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00972,"completed_at":"2026-09-06T16:15:36.635Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.428830] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0a2940f6a6d33e6840d3522","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0a2940f6a6d33e6840d3522: start","job_status":"start","queue_duration_s":2.40572,"scheduling_latency_s":2.405734} gitlab # [ 148.429884] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0a2940f6a6d33e6840d3522","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0a2940f6a6d33e6840d3522: done: 0.009553 sec","job_status":"done","queue_duration_s":2.40572,"scheduling_latency_s":2.405734,"redis_calls":13,"redis_duration_s":0.001635,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009553,"completed_at":"2026-09-06T16:15:36.646Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.442860] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"716464e97a705d9a290c4088","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-716464e97a705d9a290c4088: start","job_status":"start","queue_duration_s":2.416145,"scheduling_latency_s":2.416159} gitlab # [ 148.454999] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"716464e97a705d9a290c4088","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-716464e97a705d9a290c4088: done: 0.009545 sec","job_status":"done","queue_duration_s":2.416145,"scheduling_latency_s":2.416159,"redis_calls":13,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009545,"completed_at":"2026-09-06T16:15:36.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.500806] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5014a1a99f600bc945ad3da","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5014a1a99f600bc945ad3da: start","job_status":"start","queue_duration_s":2.426594,"scheduling_latency_s":2.426607} gitlab # [ 148.501742] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5014a1a99f600bc945ad3da","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5014a1a99f600bc945ad3da: done: 0.009522 sec","job_status":"done","queue_duration_s":2.426594,"scheduling_latency_s":2.426607,"redis_calls":13,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009522,"completed_at":"2026-09-06T16:15:36.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.549378] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b3cfde9d449045f332c8ba1","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b3cfde9d449045f332c8ba1: start","job_status":"start","queue_duration_s":2.437347,"scheduling_latency_s":2.437361} gitlab # [ 148.550245] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b3cfde9d449045f332c8ba1","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b3cfde9d449045f332c8ba1: done: 0.00943 sec","job_status":"done","queue_duration_s":2.437347,"scheduling_latency_s":2.437361,"redis_calls":13,"redis_duration_s":0.001664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00943,"completed_at":"2026-09-06T16:15:36.677Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.591418] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8d40280b4f6fa59c14cbb4c","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8d40280b4f6fa59c14cbb4c: start","job_status":"start","queue_duration_s":2.447641,"scheduling_latency_s":2.447656} gitlab # [ 148.592373] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8d40280b4f6fa59c14cbb4c","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8d40280b4f6fa59c14cbb4c: done: 0.009482 sec","job_status":"done","queue_duration_s":2.447641,"scheduling_latency_s":2.447656,"redis_calls":13,"redis_duration_s":0.001616,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009482,"completed_at":"2026-09-06T16:15:36.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.648837] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e9dc09283a999788d853b6c","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e9dc09283a999788d853b6c: start","job_status":"start","queue_duration_s":2.457947,"scheduling_latency_s":2.45796} gitlab # [ 148.649796] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e9dc09283a999788d853b6c","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e9dc09283a999788d853b6c: done: 0.012977 sec","job_status":"done","queue_duration_s":2.457947,"scheduling_latency_s":2.45796,"redis_calls":13,"redis_duration_s":0.002056,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001337,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012977,"completed_at":"2026-09-06T16:15:36.701Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.694204] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9715947d54f9694b318cb83","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9715947d54f9694b318cb83: start","job_status":"start","queue_duration_s":2.471913,"scheduling_latency_s":2.471927} gitlab # [ 148.695067] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9715947d54f9694b318cb83","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9715947d54f9694b318cb83: done: 0.010349 sec","job_status":"done","queue_duration_s":2.471913,"scheduling_latency_s":2.471927,"redis_calls":13,"redis_duration_s":0.0017079999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010349,"completed_at":"2026-09-06T16:15:36.713Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.735766] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ea6254e4d0b76ec0de63745","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ea6254e4d0b76ec0de63745: start","job_status":"start","queue_duration_s":2.483179,"scheduling_latency_s":2.483194} gitlab # [ 148.736161] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ea6254e4d0b76ec0de63745","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ea6254e4d0b76ec0de63745: done: 0.01 sec","job_status":"done","queue_duration_s":2.483179,"scheduling_latency_s":2.483194,"redis_calls":13,"redis_duration_s":0.001679,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01,"completed_at":"2026-09-06T16:15:36.724Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.794170] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"290db49e089764f87477fd41","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-290db49e089764f87477fd41: start","job_status":"start","queue_duration_s":2.494052,"scheduling_latency_s":2.494077} gitlab # [ 148.795057] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"290db49e089764f87477fd41","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-290db49e089764f87477fd41: done: 0.011378 sec","job_status":"done","queue_duration_s":2.494052,"scheduling_latency_s":2.494077,"redis_calls":13,"redis_duration_s":0.002892,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011378,"completed_at":"2026-09-06T16:15:36.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.839646] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b16eb01b2618d1643fb5c7fa","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b16eb01b2618d1643fb5c7fa: start","job_status":"start","queue_duration_s":2.506525,"scheduling_latency_s":2.506539} gitlab # [ 148.840445] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b16eb01b2618d1643fb5c7fa","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b16eb01b2618d1643fb5c7fa: done: 0.010424 sec","job_status":"done","queue_duration_s":2.506525,"scheduling_latency_s":2.506539,"redis_calls":13,"redis_duration_s":0.002329,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000722,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001607,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010424,"completed_at":"2026-09-06T16:15:36.747Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.884308] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be6a511db1b9a1204b6d423c","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be6a511db1b9a1204b6d423c: start","job_status":"start","queue_duration_s":2.518409,"scheduling_latency_s":2.518423} gitlab # [ 148.885315] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be6a511db1b9a1204b6d423c","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be6a511db1b9a1204b6d423c: done: 0.010426 sec","job_status":"done","queue_duration_s":2.518409,"scheduling_latency_s":2.518423,"redis_calls":13,"redis_duration_s":0.002737,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010426,"completed_at":"2026-09-06T16:15:36.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.931524] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17acf92ad902dd919fb13626","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17acf92ad902dd919fb13626: start","job_status":"start","queue_duration_s":2.529898,"scheduling_latency_s":2.529912} gitlab # [ 148.932428] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17acf92ad902dd919fb13626","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17acf92ad902dd919fb13626: done: 0.010573 sec","job_status":"done","queue_duration_s":2.529898,"scheduling_latency_s":2.529912,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010573,"completed_at":"2026-09-06T16:15:36.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.951262] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9689650722e075001d9f30f5","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9689650722e075001d9f30f5: start","job_status":"start","queue_duration_s":2.541578,"scheduling_latency_s":2.541591} gitlab # [ 148.994304] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9689650722e075001d9f30f5","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9689650722e075001d9f30f5: done: 0.010362 sec","job_status":"done","queue_duration_s":2.541578,"scheduling_latency_s":2.541591,"redis_calls":13,"redis_duration_s":0.002281,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001584,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010362,"completed_at":"2026-09-06T16:15:36.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.997088] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e8d05d6ebba4993a3f259bb","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e8d05d6ebba4993a3f259bb: start","job_status":"start","queue_duration_s":2.552935,"scheduling_latency_s":2.552949} gitlab # [ 149.037946] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e8d05d6ebba4993a3f259bb","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e8d05d6ebba4993a3f259bb: done: 0.01033 sec","job_status":"done","queue_duration_s":2.552935,"scheduling_latency_s":2.552949,"redis_calls":13,"redis_duration_s":0.0022979999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001589,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01033,"completed_at":"2026-09-06T16:15:36.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.044594] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0b1f7eb6494e18963b09043","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0b1f7eb6494e18963b09043: start","job_status":"start","queue_duration_s":2.628006,"scheduling_latency_s":2.628067} gitlab # [ 149.056681] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0b1f7eb6494e18963b09043","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0b1f7eb6494e18963b09043: done: 0.013617 sec","job_status":"done","queue_duration_s":2.628006,"scheduling_latency_s":2.628067,"redis_calls":13,"redis_duration_s":0.002372,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013617,"completed_at":"2026-09-06T16:15:36.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.104026] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c16017f0ad96575b7933ba0","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c16017f0ad96575b7933ba0: start","job_status":"start","queue_duration_s":2.642695,"scheduling_latency_s":2.642709} gitlab # [ 149.104653] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c16017f0ad96575b7933ba0","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c16017f0ad96575b7933ba0: done: 0.010388 sec","job_status":"done","queue_duration_s":2.642695,"scheduling_latency_s":2.642709,"redis_calls":13,"redis_duration_s":0.002298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001546,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010388,"completed_at":"2026-09-06T16:15:36.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.137843] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b45d4a8e0c5bf1d529ecdcd9","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b45d4a8e0c5bf1d529ecdcd9: start","job_status":"start","queue_duration_s":2.654112,"scheduling_latency_s":2.654126} gitlab # [ 149.182334] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b45d4a8e0c5bf1d529ecdcd9","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b45d4a8e0c5bf1d529ecdcd9: done: 0.010368 sec","job_status":"done","queue_duration_s":2.654112,"scheduling_latency_s":2.654126,"redis_calls":13,"redis_duration_s":0.0023090000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001531,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010368,"completed_at":"2026-09-06T16:15:36.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.183669] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edafd72af8fd4847fb86c0e5","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edafd72af8fd4847fb86c0e5: start","job_status":"start","queue_duration_s":2.665504,"scheduling_latency_s":2.665518} gitlab # [ 149.194892] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edafd72af8fd4847fb86c0e5","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edafd72af8fd4847fb86c0e5: done: 0.010607 sec","job_status":"done","queue_duration_s":2.665504,"scheduling_latency_s":2.665518,"redis_calls":13,"redis_duration_s":0.002272,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001577,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010607,"completed_at":"2026-09-06T16:15:36.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.266643] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0045880533b3ba7e775d431d","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0045880533b3ba7e775d431d: start","job_status":"start","queue_duration_s":2.677124,"scheduling_latency_s":2.677137} gitlab # [ 149.267518] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0045880533b3ba7e775d431d","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0045880533b3ba7e775d431d: done: 0.010218 sec","job_status":"done","queue_duration_s":2.677124,"scheduling_latency_s":2.677137,"redis_calls":13,"redis_duration_s":0.0023020000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001539,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010218,"completed_at":"2026-09-06T16:15:36.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.308139] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92c0e5c26614d76884430e87","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92c0e5c26614d76884430e87: start","job_status":"start","queue_duration_s":2.688347,"scheduling_latency_s":2.688361} gitlab # [ 149.308839] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92c0e5c26614d76884430e87","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92c0e5c26614d76884430e87: done: 0.010393 sec","job_status":"done","queue_duration_s":2.688347,"scheduling_latency_s":2.688361,"redis_calls":13,"redis_duration_s":0.002241,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000698,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010393,"completed_at":"2026-09-06T16:15:36.929Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.319603] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a02e11566c43351652cda938","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a02e11566c43351652cda938: start","job_status":"start","queue_duration_s":2.699722,"scheduling_latency_s":2.699735} gitlab # [ 149.330554] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a02e11566c43351652cda938","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a02e11566c43351652cda938: done: 0.010486 sec","job_status":"done","queue_duration_s":2.699722,"scheduling_latency_s":2.699735,"redis_calls":13,"redis_duration_s":0.002267,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001552,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010486,"completed_at":"2026-09-06T16:15:36.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.375097] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76930a1a6ad161af650c10cb","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76930a1a6ad161af650c10cb: start","job_status":"start","queue_duration_s":2.71186,"scheduling_latency_s":2.711873} gitlab # [ 149.375814] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76930a1a6ad161af650c10cb","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76930a1a6ad161af650c10cb: done: 0.010783 sec","job_status":"done","queue_duration_s":2.71186,"scheduling_latency_s":2.711873,"redis_calls":13,"redis_duration_s":0.00233,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010783,"completed_at":"2026-09-06T16:15:36.953Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.422739] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200df8460a9471cb079c4794","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200df8460a9471cb079c4794: start","job_status":"start","queue_duration_s":2.723665,"scheduling_latency_s":2.723679} gitlab # [ 149.423601] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200df8460a9471cb079c4794","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200df8460a9471cb079c4794: done: 0.010356 sec","job_status":"done","queue_duration_s":2.723665,"scheduling_latency_s":2.723679,"redis_calls":13,"redis_duration_s":0.002221,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001539,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010356,"completed_at":"2026-09-06T16:15:36.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.487927] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2202fe1c6c6397752f86ba49","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2202fe1c6c6397752f86ba49: start","job_status":"start","queue_duration_s":2.735143,"scheduling_latency_s":2.735156} gitlab # [ 149.488744] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2202fe1c6c6397752f86ba49","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2202fe1c6c6397752f86ba49: done: 0.01032 sec","job_status":"done","queue_duration_s":2.735143,"scheduling_latency_s":2.735156,"redis_calls":13,"redis_duration_s":0.002386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001588,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01032,"completed_at":"2026-09-06T16:15:36.976Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.527939] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2682fade5bf59d18d0adfb77","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2682fade5bf59d18d0adfb77: start","job_status":"start","queue_duration_s":2.746489,"scheduling_latency_s":2.746503} gitlab # [ 149.528648] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2682fade5bf59d18d0adfb77","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2682fade5bf59d18d0adfb77: done: 0.010683 sec","job_status":"done","queue_duration_s":2.746489,"scheduling_latency_s":2.746503,"redis_calls":13,"redis_duration_s":0.002253,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000699,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001554,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010683,"completed_at":"2026-09-06T16:15:36.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.576179] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:36.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abdc16ead8a31719ff3115c6","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abdc16ead8a31719ff3115c6: start","job_status":"start","queue_duration_s":2.758236,"scheduling_latency_s":2.75825} gitlab # [ 149.576822] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abdc16ead8a31719ff3115c6","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abdc16ead8a31719ff3115c6: done: 0.020725 sec","job_status":"done","queue_duration_s":2.758236,"scheduling_latency_s":2.75825,"redis_calls":13,"redis_duration_s":0.008551,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007793,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020725,"completed_at":"2026-09-06T16:15:37.009Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.585660] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"108775392febf39589f4aac3","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-108775392febf39589f4aac3: start","job_status":"start","queue_duration_s":2.780336,"scheduling_latency_s":2.78036} gitlab # [ 149.596828] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"108775392febf39589f4aac3","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-108775392febf39589f4aac3: done: 0.011866 sec","job_status":"done","queue_duration_s":2.780336,"scheduling_latency_s":2.78036,"redis_calls":13,"redis_duration_s":0.00178,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011866,"completed_at":"2026-09-06T16:15:37.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.641735] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb3df8cf90486cb033ab3e07","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb3df8cf90486cb033ab3e07: start","job_status":"start","queue_duration_s":2.793105,"scheduling_latency_s":2.793119} gitlab # [ 149.642585] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb3df8cf90486cb033ab3e07","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb3df8cf90486cb033ab3e07: done: 0.009866 sec","job_status":"done","queue_duration_s":2.793105,"scheduling_latency_s":2.793119,"redis_calls":13,"redis_duration_s":0.001764,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009866,"completed_at":"2026-09-06T16:15:37.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.704680] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e7b5b015d46a425b8e4406b","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e7b5b015d46a425b8e4406b: start","job_status":"start","queue_duration_s":2.803963,"scheduling_latency_s":2.803977} gitlab # [ 149.705600] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e7b5b015d46a425b8e4406b","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e7b5b015d46a425b8e4406b: done: 0.016409 sec","job_status":"done","queue_duration_s":2.803963,"scheduling_latency_s":2.803977,"redis_calls":13,"redis_duration_s":0.0036769999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002809,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016409,"completed_at":"2026-09-06T16:15:37.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.742837] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dea21b95905c5a6121cddd6","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dea21b95905c5a6121cddd6: start","job_status":"start","queue_duration_s":2.82241,"scheduling_latency_s":2.822444} gitlab # [ 149.743712] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dea21b95905c5a6121cddd6","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dea21b95905c5a6121cddd6: done: 0.018427 sec","job_status":"done","queue_duration_s":2.82241,"scheduling_latency_s":2.822444,"redis_calls":13,"redis_duration_s":0.0024100000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001628,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018427,"completed_at":"2026-09-06T16:15:37.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.762566] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6997164ee8b2bd8fe218a102","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6997164ee8b2bd8fe218a102: start","job_status":"start","queue_duration_s":2.842044,"scheduling_latency_s":2.842066} gitlab # [ 149.799704] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6997164ee8b2bd8fe218a102","created_at":"2026-09-06T16:15:34.081Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-bb81038c04d54257","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"23f674a4c54a348af10d622aa86b4c8b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.230Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6997164ee8b2bd8fe218a102: done: 0.009374 sec","job_status":"done","queue_duration_s":2.842044,"scheduling_latency_s":2.842066,"redis_calls":13,"redis_duration_s":0.001681,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009374,"completed_at":"2026-09-06T16:15:37.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.840829] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.083Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"67c233c6e8613df4fabe84c3","created_at":"2026-09-06T16:15:34.226Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8b339f491d7f741bef120d02e4d4b00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.236Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-67c233c6e8613df4fabe84c3: start","job_status":"start","queue_duration_s":2.846354,"scheduling_latency_s":2.846367} gitlab # [ 149.841658] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.091Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 149.842183] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.095Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"67c233c6e8613df4fabe84c3","created_at":"2026-09-06T16:15:34.226Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8b339f491d7f741bef120d02e4d4b00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:15:34.236Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-67c233c6e8613df4fabe84c3: done: 0.012288 sec","job_status":"done","queue_duration_s":2.846354,"scheduling_latency_s":2.846367,"redis_calls":10,"redis_duration_s":0.001473,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000329,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000230476,"duration_s":0.012288,"completed_at":"2026-09-06T16:15:37.095Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.891263] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6e4952ee92035361b06614f9","created_at":"2026-09-06T16:15:34.244Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf9b9c961ceeaec450b0b9adbfce75cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:15:34.250Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6e4952ee92035361b06614f9: start","job_status":"start","queue_duration_s":2.846324,"scheduling_latency_s":2.846337} gitlab # [ 149.891935] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.103Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 149.892603] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:15:37.107Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6e4952ee92035361b06614f9","created_at":"2026-09-06T16:15:34.244Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf9b9c961ceeaec450b0b9adbfce75cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:15:34.250Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6e4952ee92035361b06614f9: done: 0.011025 sec","job_status":"done","queue_duration_s":2.846324,"scheduling_latency_s":2.846337,"redis_calls":10,"redis_duration_s":0.001386,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207009,"duration_s":0.011025,"completed_at":"2026-09-06T16:15:37.107Z","load_balancing_strategy":"primary","db_duration_s":0.000291,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.095406] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1674523345 wd_nsec: 1674523047 gitlab # [ 166.560656] setup-gitlab-start[2246]: W, [2026-09-06T16:15:55.593612 #2246] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 177.269177] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.301Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"634054c477239debcb47c2b3","created_at":"2026-09-06T16:16:06.249Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.298Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-634054c477239debcb47c2b3: start","job_status":"start","queue_duration_s":0.003218,"scheduling_latency_s":0.003274} gitlab # [ 177.609403] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"634054c477239debcb47c2b3","created_at":"2026-09-06T16:16:06.249Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.298Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-634054c477239debcb47c2b3: done: 0.341 sec","job_status":"done","queue_duration_s":0.003218,"scheduling_latency_s":0.003274,"redis_calls":66,"redis_duration_s":0.074185,"redis_read_bytes":118,"redis_write_bytes":41336,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.01398,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.001861,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":57,"redis_queues_metadata_duration_s":0.055653,"redis_queues_metadata_read_bytes":108,"redis_queues_metadata_write_bytes":3378,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002691,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.125088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000851784,"duration_s":0.341,"completed_at":"2026-09-06T16:16:06.642Z","load_balancing_strategy":"primary","db_duration_s":0.030106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.613678] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.647Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c3d8ec9086e7f6c7fe4d990f","created_at":"2026-09-06T16:16:06.311Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebf1931828771d1986b1804a5aecbc33","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.314Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c3d8ec9086e7f6c7fe4d990f: start","job_status":"start","queue_duration_s":0.332601,"scheduling_latency_s":0.332632} gitlab # [ 177.666145] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c3d8ec9086e7f6c7fe4d990f","created_at":"2026-09-06T16:16:06.311Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebf1931828771d1986b1804a5aecbc33","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.314Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c3d8ec9086e7f6c7fe4d990f: done: 0.052513 sec","job_status":"done","queue_duration_s":0.332601,"scheduling_latency_s":0.332632,"redis_calls":7,"redis_duration_s":0.006409,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001285,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052513,"completed_at":"2026-09-06T16:16:06.699Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.669937] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"97fde7d01773c90504982a0e","created_at":"2026-09-06T16:16:06.330Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"327e54c779ed9d2cd5f149c23be56196","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:16:06.337Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-97fde7d01773c90504982a0e: start","job_status":"start","queue_duration_s":0.36654,"scheduling_latency_s":0.366558} gitlab # [ 177.685621] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.719Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 177.695704] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"97fde7d01773c90504982a0e","created_at":"2026-09-06T16:16:06.330Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"327e54c779ed9d2cd5f149c23be56196","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:16:06.337Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-97fde7d01773c90504982a0e: done: 0.025685 sec","job_status":"done","queue_duration_s":0.36654,"scheduling_latency_s":0.366558,"redis_calls":7,"redis_duration_s":0.002502,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001417,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025685,"completed_at":"2026-09-06T16:16:06.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.746382] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.732Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"488382538db1cd21d906438c","created_at":"2026-09-06T16:16:06.342Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.344Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-488382538db1cd21d906438c: start","job_status":"start","queue_duration_s":0.387874,"scheduling_latency_s":0.387906} gitlab # [ 177.747236] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"488382538db1cd21d906438c","created_at":"2026-09-06T16:16:06.342Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.344Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-488382538db1cd21d906438c: done: 0.025351 sec","job_status":"done","queue_duration_s":0.387874,"scheduling_latency_s":0.387906,"redis_calls":14,"redis_duration_s":0.004201,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000279,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002921,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025351,"completed_at":"2026-09-06T16:16:06.757Z","load_balancing_strategy":"primary","db_duration_s":0.001023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.762250] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"572020ffefa778550e2d2a31","created_at":"2026-09-06T16:16:06.360Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.361Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-572020ffefa778550e2d2a31: start","job_status":"start","queue_duration_s":0.397267,"scheduling_latency_s":0.397283} gitlab # [ 177.831354] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.864Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"572020ffefa778550e2d2a31","created_at":"2026-09-06T16:16:06.360Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.361Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-572020ffefa778550e2d2a31: done: 0.105428 sec","job_status":"done","queue_duration_s":0.397267,"scheduling_latency_s":0.397283,"redis_calls":60,"redis_duration_s":0.011815,"redis_read_bytes":109,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.0006,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.010607,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000608,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.105428,"completed_at":"2026-09-06T16:16:06.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.868309] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a67533b4278317c15b264e08","created_at":"2026-09-06T16:16:06.384Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"470cf107465520f7c88a2e69bc845854","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.388Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a67533b4278317c15b264e08: start","job_status":"start","queue_duration_s":0.512859,"scheduling_latency_s":0.512887} gitlab # [ 177.875103] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.908Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 177.879242] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.912Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a67533b4278317c15b264e08","created_at":"2026-09-06T16:16:06.384Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"470cf107465520f7c88a2e69bc845854","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.388Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a67533b4278317c15b264e08: done: 0.011221 sec","job_status":"done","queue_duration_s":0.512859,"scheduling_latency_s":0.512887,"redis_calls":7,"redis_duration_s":0.001134,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000371,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011221,"completed_at":"2026-09-06T16:16:06.912Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.881086] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.913Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"7bed4d48dd200875d493adc3","created_at":"2026-09-06T16:16:06.395Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d605bf18bc5b051f7edf49ede33ef38f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.399Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-7bed4d48dd200875d493adc3: start","job_status":"start","queue_duration_s":0.514403,"scheduling_latency_s":0.51442} gitlab # [ 177.894950] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"7bed4d48dd200875d493adc3","created_at":"2026-09-06T16:16:06.395Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d605bf18bc5b051f7edf49ede33ef38f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.399Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-7bed4d48dd200875d493adc3: done: 0.012664 sec","job_status":"done","queue_duration_s":0.514403,"scheduling_latency_s":0.51442,"redis_calls":9,"redis_duration_s":0.0012920000000000002,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00035647,"exclusive_lock_hold_duration_s":0.002946743,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.012664,"completed_at":"2026-09-06T16:16:06.926Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.933164] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.927Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"278425ac1791cb4bf01b863f","created_at":"2026-09-06T16:16:06.418Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f240f357785a58c7ce89029adf0e0d3","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.433Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-278425ac1791cb4bf01b863f: start","job_status":"start","queue_duration_s":0.49432,"scheduling_latency_s":0.494336} gitlab # [ 177.972130] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.942Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"278425ac1791cb4bf01b863f","created_at":"2026-09-06T16:16:06.418Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f240f357785a58c7ce89029adf0e0d3","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.433Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-278425ac1791cb4bf01b863f: done: 0.015084 sec","job_status":"done","queue_duration_s":0.49432,"scheduling_latency_s":0.494336,"redis_calls":12,"redis_duration_s":0.001834,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000505,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000221536,"duration_s":0.015084,"completed_at":"2026-09-06T16:16:06.942Z","load_balancing_strategy":"primary","db_duration_s":0.001313,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.011053] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.943Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cc53c1ac9178dc4a1491a774","created_at":"2026-09-06T16:16:06.441Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da0cdfad848b7c696c7f411256c0c374","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.444Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cc53c1ac9178dc4a1491a774: start","job_status":"start","queue_duration_s":0.4988,"scheduling_latency_s":0.498814} gitlab # [ 178.011742] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cc53c1ac9178dc4a1491a774","created_at":"2026-09-06T16:16:06.441Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da0cdfad848b7c696c7f411256c0c374","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.444Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cc53c1ac9178dc4a1491a774: done: 0.010304 sec","job_status":"done","queue_duration_s":0.4988,"scheduling_latency_s":0.498814,"redis_calls":7,"redis_duration_s":0.0010119999999999999,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010304,"completed_at":"2026-09-06T16:16:06.953Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.062312] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.955Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"78d9574e53e7dad841af6df7","created_at":"2026-09-06T16:16:06.453Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"96d3530e105da818af571765b46a17ed","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.462Z","job_size_bytes":3,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-78d9574e53e7dad841af6df7: start","job_status":"start","queue_duration_s":0.492793,"scheduling_latency_s":0.492807} gitlab # [ 178.062954] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.968Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"78d9574e53e7dad841af6df7","created_at":"2026-09-06T16:16:06.453Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"96d3530e105da818af571765b46a17ed","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.462Z","job_size_bytes":3,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-78d9574e53e7dad841af6df7: done: 0.012766 sec","job_status":"done","queue_duration_s":0.492793,"scheduling_latency_s":0.492807,"redis_calls":10,"redis_duration_s":0.001353,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000298,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000236902,"duration_s":0.012766,"completed_at":"2026-09-06T16:16:06.968Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.108148] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d573c3445cecaa75e739921b","created_at":"2026-09-06T16:16:06.473Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90efdc76b7196297a3c239865a3ff5c0","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.482Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d573c3445cecaa75e739921b: start","job_status":"start","queue_duration_s":0.486341,"scheduling_latency_s":0.486356} gitlab # [ 178.108905] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.992Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d573c3445cecaa75e739921b","created_at":"2026-09-06T16:16:06.473Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90efdc76b7196297a3c239865a3ff5c0","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.482Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d573c3445cecaa75e739921b: done: 0.023031 sec","job_status":"done","queue_duration_s":0.486341,"scheduling_latency_s":0.486356,"redis_calls":10,"redis_duration_s":0.001603,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000301,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000284673,"duration_s":0.023031,"completed_at":"2026-09-06T16:16:06.992Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003194,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.146832] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:06.994Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"eddba65755020e7a2dbd24da","created_at":"2026-09-06T16:16:06.490Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cd99711f472888da239e6a8fdf6184e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.493Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-eddba65755020e7a2dbd24da: start","job_status":"start","queue_duration_s":0.500357,"scheduling_latency_s":0.500372} gitlab # [ 178.147687] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.023Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"eddba65755020e7a2dbd24da","created_at":"2026-09-06T16:16:06.490Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cd99711f472888da239e6a8fdf6184e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.493Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-eddba65755020e7a2dbd24da: done: 0.029625 sec","job_status":"done","queue_duration_s":0.500357,"scheduling_latency_s":0.500372,"redis_calls":9,"redis_duration_s":0.0015960000000000002,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000796,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000406197,"exclusive_lock_hold_duration_s":0.014017703,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.029625,"completed_at":"2026-09-06T16:16:07.023Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.194385] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f97812c340e3a9312e891473","created_at":"2026-09-06T16:16:06.504Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f71be22cca21b4f4982e8a6906242063","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:16:06.507Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f97812c340e3a9312e891473: start","job_status":"start","queue_duration_s":0.51763,"scheduling_latency_s":0.517645} gitlab # [ 178.195138] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f97812c340e3a9312e891473","created_at":"2026-09-06T16:16:06.504Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f71be22cca21b4f4982e8a6906242063","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:16:06.507Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f97812c340e3a9312e891473: done: 0.014846 sec","job_status":"done","queue_duration_s":0.51763,"scheduling_latency_s":0.517645,"redis_calls":7,"redis_duration_s":0.001036,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014846,"completed_at":"2026-09-06T16:16:07.039Z","load_balancing_strategy":"primary","db_duration_s":0.001177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.234994] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.040Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1bcedbb0088355ad08dba35c","created_at":"2026-09-06T16:16:06.514Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89b39b9cb6ff6d9a47d4cb9fc6c18aea","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.519Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1bcedbb0088355ad08dba35c: start","job_status":"start","queue_duration_s":0.521056,"scheduling_latency_s":0.521071} gitlab # [ 178.235745] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.057Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1bcedbb0088355ad08dba35c","created_at":"2026-09-06T16:16:06.514Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89b39b9cb6ff6d9a47d4cb9fc6c18aea","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.519Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1bcedbb0088355ad08dba35c: done: 0.016943 sec","job_status":"done","queue_duration_s":0.521056,"scheduling_latency_s":0.521071,"redis_calls":10,"redis_duration_s":0.001565,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00066,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004905651,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.016943,"completed_at":"2026-09-06T16:16:07.057Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001932,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 178.295516] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.058Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"a8b0857f31b802684582e1b1","created_at":"2026-09-06T16:16:06.529Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26744b1fa00c8c21ca4da81029a3430f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.533Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-a8b0857f31b802684582e1b1: start","job_status":"start","queue_duration_s":0.524945,"scheduling_latency_s":0.524962} gitlab # [ 178.359368] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"a8b0857f31b802684582e1b1","created_at":"2026-09-06T16:16:06.529Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26744b1fa00c8c21ca4da81029a3430f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.533Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-a8b0857f31b802684582e1b1: done: 0.334002 sec","job_status":"done","queue_duration_s":0.524945,"scheduling_latency_s":0.524962,"redis_calls":9,"redis_duration_s":0.00174,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000467936,"exclusive_lock_hold_duration_s":0.320876815,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.334002,"completed_at":"2026-09-06T16:16:07.392Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.314184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.361249] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"77e6a86b8c899c26adaf36f6","created_at":"2026-09-06T16:16:06.543Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"433c9ed46e0d0a3bead99b39754f6e41","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-77e6a86b8c899c26adaf36f6: start","job_status":"start","queue_duration_s":0.846007,"scheduling_latency_s":0.846023} gitlab # [ 178.372816] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.399Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 178.376089] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"77e6a86b8c899c26adaf36f6","created_at":"2026-09-06T16:16:06.543Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"433c9ed46e0d0a3bead99b39754f6e41","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-77e6a86b8c899c26adaf36f6: done: 0.009001 sec","job_status":"done","queue_duration_s":0.846007,"scheduling_latency_s":0.846023,"redis_calls":7,"redis_duration_s":0.001005,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009001,"completed_at":"2026-09-06T16:16:07.403Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.431373] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c141c8c0f1a2729abd7723d2","created_at":"2026-09-06T16:16:06.584Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1596b900173dda4c7300479ca92b85f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.589Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c141c8c0f1a2729abd7723d2: start","job_status":"start","queue_duration_s":0.814858,"scheduling_latency_s":0.814872} gitlab # [ 178.432132] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c141c8c0f1a2729abd7723d2","created_at":"2026-09-06T16:16:06.584Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1596b900173dda4c7300479ca92b85f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.589Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c141c8c0f1a2729abd7723d2: done: 0.008241 sec","job_status":"done","queue_duration_s":0.814858,"scheduling_latency_s":0.814872,"redis_calls":7,"redis_duration_s":0.000915,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008241,"completed_at":"2026-09-06T16:16:07.412Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.482675] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.413Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"405f2df996102ef37373326a","created_at":"2026-09-06T16:16:06.603Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"413eeb03443e00c5db5fb1d755a74bed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.607Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-405f2df996102ef37373326a: start","job_status":"start","queue_duration_s":0.806177,"scheduling_latency_s":0.806191} gitlab # [ 178.483739] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"405f2df996102ef37373326a","created_at":"2026-09-06T16:16:06.603Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"413eeb03443e00c5db5fb1d755a74bed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.607Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-405f2df996102ef37373326a: done: 0.00839 sec","job_status":"done","queue_duration_s":0.806177,"scheduling_latency_s":0.806191,"redis_calls":7,"redis_duration_s":0.00091,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.00839,"completed_at":"2026-09-06T16:16:07.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.526867] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a71fbc5a41b2dbf8e9742ad5","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a71fbc5a41b2dbf8e9742ad5: start","job_status":"start","queue_duration_s":0.804018,"scheduling_latency_s":0.804032} gitlab # [ 178.602126] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a71fbc5a41b2dbf8e9742ad5","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a71fbc5a41b2dbf8e9742ad5: done: 0.212103 sec","job_status":"done","queue_duration_s":0.804018,"scheduling_latency_s":0.804032,"redis_calls":13,"redis_duration_s":0.004066,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000744,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.17,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.168,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000547555,"duration_s":0.212103,"completed_at":"2026-09-06T16:16:07.635Z","load_balancing_strategy":"primary","db_duration_s":0.175619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.604359] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d43ddbc85ce94fb8bd16d57f","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d43ddbc85ce94fb8bd16d57f: start","job_status":"start","queue_duration_s":1.018667,"scheduling_latency_s":1.018702} gitlab # [ 178.630078] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d43ddbc85ce94fb8bd16d57f","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d43ddbc85ce94fb8bd16d57f: done: 0.025681 sec","job_status":"done","queue_duration_s":1.018667,"scheduling_latency_s":1.018702,"redis_calls":10,"redis_duration_s":0.003898,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001944,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025681,"completed_at":"2026-09-06T16:16:07.663Z","load_balancing_strategy":"primary","db_duration_s":0.002358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.684092] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a7a5052fe9483b8897b49e0e","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a7a5052fe9483b8897b49e0e: start","job_status":"start","queue_duration_s":1.046106,"scheduling_latency_s":1.046133} gitlab # [ 178.684938] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a7a5052fe9483b8897b49e0e","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a7a5052fe9483b8897b49e0e: done: 0.01645 sec","job_status":"done","queue_duration_s":1.046106,"scheduling_latency_s":1.046133,"redis_calls":10,"redis_duration_s":0.002015,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01645,"completed_at":"2026-09-06T16:16:07.681Z","load_balancing_strategy":"primary","db_duration_s":0.004853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.731989] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a10d4a834ca3a5ac51f1459d","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a10d4a834ca3a5ac51f1459d: start","job_status":"start","queue_duration_s":1.063679,"scheduling_latency_s":1.063693} gitlab # [ 178.732939] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a10d4a834ca3a5ac51f1459d","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a10d4a834ca3a5ac51f1459d: done: 0.013851 sec","job_status":"done","queue_duration_s":1.063679,"scheduling_latency_s":1.063693,"redis_calls":10,"redis_duration_s":0.002141,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001289,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013851,"completed_at":"2026-09-06T16:16:07.696Z","load_balancing_strategy":"primary","db_duration_s":0.001759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.744201] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"920f9d41279d5f9fbe4966d8","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-920f9d41279d5f9fbe4966d8: start","job_status":"start","queue_duration_s":1.078634,"scheduling_latency_s":1.078648} gitlab # [ 178.792442] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"920f9d41279d5f9fbe4966d8","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-920f9d41279d5f9fbe4966d8: done: 0.013513 sec","job_status":"done","queue_duration_s":1.078634,"scheduling_latency_s":1.078648,"redis_calls":10,"redis_duration_s":0.002482,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001597,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013513,"completed_at":"2026-09-06T16:16:07.711Z","load_balancing_strategy":"primary","db_duration_s":0.001766,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.794137] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1ff24cc6f17b5b9f8097aa78","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1ff24cc6f17b5b9f8097aa78: start","job_status":"start","queue_duration_s":1.093335,"scheduling_latency_s":1.09335} gitlab # [ 178.842954] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1ff24cc6f17b5b9f8097aa78","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1ff24cc6f17b5b9f8097aa78: done: 0.016076 sec","job_status":"done","queue_duration_s":1.093335,"scheduling_latency_s":1.09335,"redis_calls":10,"redis_duration_s":0.001981,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016076,"completed_at":"2026-09-06T16:16:07.728Z","load_balancing_strategy":"primary","db_duration_s":0.00639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.844095] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7783fb579b664107d21bd455","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7783fb579b664107d21bd455: start","job_status":"start","queue_duration_s":1.110619,"scheduling_latency_s":1.110632} gitlab # [ 178.851886] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7783fb579b664107d21bd455","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7783fb579b664107d21bd455: done: 0.013375 sec","job_status":"done","queue_duration_s":1.110619,"scheduling_latency_s":1.110632,"redis_calls":10,"redis_duration_s":0.002319,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001459,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013375,"completed_at":"2026-09-06T16:16:07.743Z","load_balancing_strategy":"primary","db_duration_s":0.001718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.906237] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0eb433f129735c50bf70af7e","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0eb433f129735c50bf70af7e: start","job_status":"start","queue_duration_s":1.125657,"scheduling_latency_s":1.125671} gitlab # [ 178.907200] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0eb433f129735c50bf70af7e","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0eb433f129735c50bf70af7e: done: 0.013468 sec","job_status":"done","queue_duration_s":1.125657,"scheduling_latency_s":1.125671,"redis_calls":10,"redis_duration_s":0.002352,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001492,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013468,"completed_at":"2026-09-06T16:16:07.758Z","load_balancing_strategy":"primary","db_duration_s":0.001639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.949475] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6ab6e735d9ae81f0e4367a15","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6ab6e735d9ae81f0e4367a15: start","job_status":"start","queue_duration_s":1.14023,"scheduling_latency_s":1.140245} gitlab # [ 178.950380] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6ab6e735d9ae81f0e4367a15","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6ab6e735d9ae81f0e4367a15: done: 0.017536 sec","job_status":"done","queue_duration_s":1.14023,"scheduling_latency_s":1.140245,"redis_calls":10,"redis_duration_s":0.00185,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017536,"completed_at":"2026-09-06T16:16:07.776Z","load_balancing_strategy":"primary","db_duration_s":0.009081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.997495] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4c60c83297a49baa7f66c6b","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4c60c83297a49baa7f66c6b: start","job_status":"start","queue_duration_s":1.158678,"scheduling_latency_s":1.158692} gitlab # [ 178.999298] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4c60c83297a49baa7f66c6b","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4c60c83297a49baa7f66c6b: done: 0.013961 sec","job_status":"done","queue_duration_s":1.158678,"scheduling_latency_s":1.158692,"redis_calls":10,"redis_duration_s":0.0018310000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013961,"completed_at":"2026-09-06T16:16:07.791Z","load_balancing_strategy":"primary","db_duration_s":0.003011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.032996] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1e38d149ee0eb6f557d1664c","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1e38d149ee0eb6f557d1664c: start","job_status":"start","queue_duration_s":1.173785,"scheduling_latency_s":1.173799} gitlab # [ 179.075288] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1e38d149ee0eb6f557d1664c","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1e38d149ee0eb6f557d1664c: done: 0.01343 sec","job_status":"done","queue_duration_s":1.173785,"scheduling_latency_s":1.173799,"redis_calls":10,"redis_duration_s":0.002392,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001425,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01343,"completed_at":"2026-09-06T16:16:07.806Z","load_balancing_strategy":"primary","db_duration_s":0.001816,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.121245] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2700863dd2f4661307299874","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2700863dd2f4661307299874: start","job_status":"start","queue_duration_s":1.188387,"scheduling_latency_s":1.188403} gitlab # [ 179.122223] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2700863dd2f4661307299874","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2700863dd2f4661307299874: done: 0.013897 sec","job_status":"done","queue_duration_s":1.188387,"scheduling_latency_s":1.188403,"redis_calls":10,"redis_duration_s":0.0023,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013897,"completed_at":"2026-09-06T16:16:07.821Z","load_balancing_strategy":"primary","db_duration_s":0.001797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.128990] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9e398e4ddc3dbc30a65ed306","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9e398e4ddc3dbc30a65ed306: start","job_status":"start","queue_duration_s":1.203428,"scheduling_latency_s":1.203442} gitlab # [ 179.141521] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9e398e4ddc3dbc30a65ed306","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9e398e4ddc3dbc30a65ed306: done: 0.017216 sec","job_status":"done","queue_duration_s":1.203428,"scheduling_latency_s":1.203442,"redis_calls":10,"redis_duration_s":0.001859,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017216,"completed_at":"2026-09-06T16:16:07.839Z","load_balancing_strategy":"primary","db_duration_s":0.008524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.194124] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fc4e3a404324b721d994a717","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fc4e3a404324b721d994a717: start","job_status":"start","queue_duration_s":1.221806,"scheduling_latency_s":1.221821} gitlab # [ 179.195353] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fc4e3a404324b721d994a717","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fc4e3a404324b721d994a717: done: 0.013387 sec","job_status":"done","queue_duration_s":1.221806,"scheduling_latency_s":1.221821,"redis_calls":10,"redis_duration_s":0.002291,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013387,"completed_at":"2026-09-06T16:16:07.854Z","load_balancing_strategy":"primary","db_duration_s":0.001785,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.253199] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b859515b08a2e68062b70b3d","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b859515b08a2e68062b70b3d: start","job_status":"start","queue_duration_s":1.236379,"scheduling_latency_s":1.236394} gitlab # [ 179.253837] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b859515b08a2e68062b70b3d","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b859515b08a2e68062b70b3d: done: 0.013074 sec","job_status":"done","queue_duration_s":1.236379,"scheduling_latency_s":1.236394,"redis_calls":10,"redis_duration_s":0.002277,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001369,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013074,"completed_at":"2026-09-06T16:16:07.868Z","load_balancing_strategy":"primary","db_duration_s":0.001645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.274791] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f1b07f660b4fbec5974fbbaa","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f1b07f660b4fbec5974fbbaa: start","job_status":"start","queue_duration_s":1.25111,"scheduling_latency_s":1.251124} gitlab # [ 179.326354] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f1b07f660b4fbec5974fbbaa","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f1b07f660b4fbec5974fbbaa: done: 0.01821 sec","job_status":"done","queue_duration_s":1.25111,"scheduling_latency_s":1.251124,"redis_calls":10,"redis_duration_s":0.002115,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001315,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01821,"completed_at":"2026-09-06T16:16:07.888Z","load_balancing_strategy":"primary","db_duration_s":0.010961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.327897] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"21fb5485727590eeb0767bbe","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-21fb5485727590eeb0767bbe: start","job_status":"start","queue_duration_s":1.270256,"scheduling_latency_s":1.27027} gitlab # [ 179.334950] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"21fb5485727590eeb0767bbe","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-21fb5485727590eeb0767bbe: done: 0.013831 sec","job_status":"done","queue_duration_s":1.270256,"scheduling_latency_s":1.27027,"redis_calls":10,"redis_duration_s":0.0016380000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013831,"completed_at":"2026-09-06T16:16:07.903Z","load_balancing_strategy":"primary","db_duration_s":0.003227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.386729] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9c6b68d31cee84d66b0c3864","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9c6b68d31cee84d66b0c3864: start","job_status":"start","queue_duration_s":1.285699,"scheduling_latency_s":1.285713} gitlab # [ 179.387682] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9c6b68d31cee84d66b0c3864","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9c6b68d31cee84d66b0c3864: done: 0.012951 sec","job_status":"done","queue_duration_s":1.285699,"scheduling_latency_s":1.285713,"redis_calls":10,"redis_duration_s":0.002019,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012951,"completed_at":"2026-09-06T16:16:07.917Z","load_balancing_strategy":"primary","db_duration_s":0.001707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.450160] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"76bf03597dafa942cb1ed643","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-76bf03597dafa942cb1ed643: start","job_status":"start","queue_duration_s":1.299715,"scheduling_latency_s":1.29973} gitlab # [ 179.450891] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"76bf03597dafa942cb1ed643","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-76bf03597dafa942cb1ed643: done: 0.013943 sec","job_status":"done","queue_duration_s":1.299715,"scheduling_latency_s":1.29973,"redis_calls":10,"redis_duration_s":0.002232,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001316,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013943,"completed_at":"2026-09-06T16:16:07.932Z","load_balancing_strategy":"primary","db_duration_s":0.001734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.501874] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"effd755ed024f18044e98a44","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-effd755ed024f18044e98a44: start","job_status":"start","queue_duration_s":1.314795,"scheduling_latency_s":1.314809} gitlab # [ 179.502730] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"effd755ed024f18044e98a44","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-effd755ed024f18044e98a44: done: 0.015308 sec","job_status":"done","queue_duration_s":1.314795,"scheduling_latency_s":1.314809,"redis_calls":10,"redis_duration_s":0.002311,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015308,"completed_at":"2026-09-06T16:16:07.949Z","load_balancing_strategy":"primary","db_duration_s":0.005867,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.515262] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"28fa7390977869637fc9ae0f","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-28fa7390977869637fc9ae0f: start","job_status":"start","queue_duration_s":1.331223,"scheduling_latency_s":1.331237} gitlab # [ 179.559187] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"28fa7390977869637fc9ae0f","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-28fa7390977869637fc9ae0f: done: 0.012959 sec","job_status":"done","queue_duration_s":1.331223,"scheduling_latency_s":1.331237,"redis_calls":10,"redis_duration_s":0.0022240000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012959,"completed_at":"2026-09-06T16:16:07.963Z","load_balancing_strategy":"primary","db_duration_s":0.001734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.564157] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"34620b3fc6b7f83d9383daef","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-34620b3fc6b7f83d9383daef: start","job_status":"start","queue_duration_s":1.345343,"scheduling_latency_s":1.345358} gitlab # [ 179.610180] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"34620b3fc6b7f83d9383daef","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-34620b3fc6b7f83d9383daef: done: 0.013407 sec","job_status":"done","queue_duration_s":1.345343,"scheduling_latency_s":1.345358,"redis_calls":10,"redis_duration_s":0.002355,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001403,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013407,"completed_at":"2026-09-06T16:16:07.977Z","load_balancing_strategy":"primary","db_duration_s":0.001836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.612951] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7311e39e8bb1961728d37a91","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7311e39e8bb1961728d37a91: start","job_status":"start","queue_duration_s":1.359906,"scheduling_latency_s":1.35992} gitlab # [ 179.655764] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7311e39e8bb1961728d37a91","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7311e39e8bb1961728d37a91: done: 0.016576 sec","job_status":"done","queue_duration_s":1.359906,"scheduling_latency_s":1.35992,"redis_calls":10,"redis_duration_s":0.002114,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016576,"completed_at":"2026-09-06T16:16:07.995Z","load_balancing_strategy":"primary","db_duration_s":0.006876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.699177] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:07.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8735fe4c65df885a22bcbd99","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8735fe4c65df885a22bcbd99: start","job_status":"start","queue_duration_s":1.377697,"scheduling_latency_s":1.377711} gitlab # [ 179.699836] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8735fe4c65df885a22bcbd99","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8735fe4c65df885a22bcbd99: done: 0.013507 sec","job_status":"done","queue_duration_s":1.377697,"scheduling_latency_s":1.377711,"redis_calls":10,"redis_duration_s":0.0025729999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013507,"completed_at":"2026-09-06T16:16:08.010Z","load_balancing_strategy":"primary","db_duration_s":0.001736,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.707200] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5b8a4ed5f5dc05b26d2e82cd","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5b8a4ed5f5dc05b26d2e82cd: start","job_status":"start","queue_duration_s":1.392704,"scheduling_latency_s":1.392718} gitlab # [ 179.747184] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5b8a4ed5f5dc05b26d2e82cd","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5b8a4ed5f5dc05b26d2e82cd: done: 0.01326 sec","job_status":"done","queue_duration_s":1.392704,"scheduling_latency_s":1.392718,"redis_calls":10,"redis_duration_s":0.002463,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001503,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01326,"completed_at":"2026-09-06T16:16:08.025Z","load_balancing_strategy":"primary","db_duration_s":0.001657,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.754037] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"55ac237ee8b52b019d0755ed","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-55ac237ee8b52b019d0755ed: start","job_status":"start","queue_duration_s":1.407137,"scheduling_latency_s":1.407151} gitlab # [ 179.799388] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"55ac237ee8b52b019d0755ed","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-55ac237ee8b52b019d0755ed: done: 0.013354 sec","job_status":"done","queue_duration_s":1.407137,"scheduling_latency_s":1.407151,"redis_calls":10,"redis_duration_s":0.0016449999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013354,"completed_at":"2026-09-06T16:16:08.039Z","load_balancing_strategy":"primary","db_duration_s":0.002814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.800714] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8e56b4664d5a115f056779fa","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8e56b4664d5a115f056779fa: start","job_status":"start","queue_duration_s":1.421397,"scheduling_latency_s":1.421412} gitlab # [ 179.812642] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8e56b4664d5a115f056779fa","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8e56b4664d5a115f056779fa: done: 0.014055 sec","job_status":"done","queue_duration_s":1.421397,"scheduling_latency_s":1.421412,"redis_calls":10,"redis_duration_s":0.001301,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014055,"completed_at":"2026-09-06T16:16:08.054Z","load_balancing_strategy":"primary","db_duration_s":0.004033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.855958] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"910c11ca90e7d0afbdd76dc2","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-910c11ca90e7d0afbdd76dc2: start","job_status":"start","queue_duration_s":1.436345,"scheduling_latency_s":1.43636} gitlab # [ 179.860167] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"910c11ca90e7d0afbdd76dc2","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-910c11ca90e7d0afbdd76dc2: done: 0.014058 sec","job_status":"done","queue_duration_s":1.436345,"scheduling_latency_s":1.43636,"redis_calls":10,"redis_duration_s":0.001397,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014058,"completed_at":"2026-09-06T16:16:08.069Z","load_balancing_strategy":"primary","db_duration_s":0.004825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.901140] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b70a06a1f2cb2645be7fe84d","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b70a06a1f2cb2645be7fe84d: start","job_status":"start","queue_duration_s":1.451401,"scheduling_latency_s":1.451416} gitlab # [ 179.946735] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b70a06a1f2cb2645be7fe84d","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b70a06a1f2cb2645be7fe84d: done: 0.011715 sec","job_status":"done","queue_duration_s":1.451401,"scheduling_latency_s":1.451416,"redis_calls":10,"redis_duration_s":0.001683,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011715,"completed_at":"2026-09-06T16:16:08.082Z","load_balancing_strategy":"primary","db_duration_s":0.001203,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.948421] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"749e911b98104c2be2dfc4d4","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-749e911b98104c2be2dfc4d4: start","job_status":"start","queue_duration_s":1.464236,"scheduling_latency_s":1.464251} gitlab # [ 179.956755] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"749e911b98104c2be2dfc4d4","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-749e911b98104c2be2dfc4d4: done: 0.012781 sec","job_status":"done","queue_duration_s":1.464236,"scheduling_latency_s":1.464251,"redis_calls":10,"redis_duration_s":0.0021939999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001362,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012781,"completed_at":"2026-09-06T16:16:08.096Z","load_balancing_strategy":"primary","db_duration_s":0.001916,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.004637] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9b5620cf9c89bc5c48d9f621","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9b5620cf9c89bc5c48d9f621: start","job_status":"start","queue_duration_s":1.478123,"scheduling_latency_s":1.478137} gitlab # [ 180.005589] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9b5620cf9c89bc5c48d9f621","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9b5620cf9c89bc5c48d9f621: done: 0.012411 sec","job_status":"done","queue_duration_s":1.478123,"scheduling_latency_s":1.478137,"redis_calls":10,"redis_duration_s":0.0021060000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012411,"completed_at":"2026-09-06T16:16:08.109Z","load_balancing_strategy":"primary","db_duration_s":0.001613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.049644] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7388635ccdc6529f07870863","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7388635ccdc6529f07870863: start","job_status":"start","queue_duration_s":1.491506,"scheduling_latency_s":1.49152} gitlab # [ 180.053879] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7388635ccdc6529f07870863","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7388635ccdc6529f07870863: done: 0.012307 sec","job_status":"done","queue_duration_s":1.491506,"scheduling_latency_s":1.49152,"redis_calls":10,"redis_duration_s":0.00195,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012307,"completed_at":"2026-09-06T16:16:08.122Z","load_balancing_strategy":"primary","db_duration_s":0.001633,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.116796] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3d46e0e186c05825061840c","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3d46e0e186c05825061840c: start","job_status":"start","queue_duration_s":1.505121,"scheduling_latency_s":1.505136} gitlab # [ 180.117720] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3d46e0e186c05825061840c","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3d46e0e186c05825061840c: done: 0.012392 sec","job_status":"done","queue_duration_s":1.505121,"scheduling_latency_s":1.505136,"redis_calls":10,"redis_duration_s":0.002148,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001301,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012392,"completed_at":"2026-09-06T16:16:08.136Z","load_balancing_strategy":"primary","db_duration_s":0.00145,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.161664] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ccf4cd494b7f8b366a0f2c78","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ccf4cd494b7f8b366a0f2c78: start","job_status":"start","queue_duration_s":1.518656,"scheduling_latency_s":1.51867} gitlab # [ 180.162680] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ccf4cd494b7f8b366a0f2c78","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ccf4cd494b7f8b366a0f2c78: done: 0.012949 sec","job_status":"done","queue_duration_s":1.518656,"scheduling_latency_s":1.51867,"redis_calls":10,"redis_duration_s":0.002283,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001384,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012949,"completed_at":"2026-09-06T16:16:08.150Z","load_balancing_strategy":"primary","db_duration_s":0.001766,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.187488] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"28bdba76ee1bc3e170c5c993","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-28bdba76ee1bc3e170c5c993: start","job_status":"start","queue_duration_s":1.532747,"scheduling_latency_s":1.532761} gitlab # [ 180.199314] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"28bdba76ee1bc3e170c5c993","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-28bdba76ee1bc3e170c5c993: done: 0.015936 sec","job_status":"done","queue_duration_s":1.532747,"scheduling_latency_s":1.532761,"redis_calls":10,"redis_duration_s":0.0014759999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000876,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015936,"completed_at":"2026-09-06T16:16:08.167Z","load_balancing_strategy":"primary","db_duration_s":0.007207,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.244094] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"09de6920b2ffe4e68dbca4f7","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-09de6920b2ffe4e68dbca4f7: start","job_status":"start","queue_duration_s":1.54965,"scheduling_latency_s":1.549666} gitlab # [ 180.247514] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"09de6920b2ffe4e68dbca4f7","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-09de6920b2ffe4e68dbca4f7: done: 0.012765 sec","job_status":"done","queue_duration_s":1.54965,"scheduling_latency_s":1.549666,"redis_calls":10,"redis_duration_s":0.0012980000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00077,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012765,"completed_at":"2026-09-06T16:16:08.181Z","load_balancing_strategy":"primary","db_duration_s":0.003166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.290639] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"53145b7e7dbce884f69d3a9e","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-53145b7e7dbce884f69d3a9e: start","job_status":"start","queue_duration_s":1.563324,"scheduling_latency_s":1.563339} gitlab # [ 180.295790] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"53145b7e7dbce884f69d3a9e","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-53145b7e7dbce884f69d3a9e: done: 0.011213 sec","job_status":"done","queue_duration_s":1.563324,"scheduling_latency_s":1.563339,"redis_calls":10,"redis_duration_s":0.001536,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011213,"completed_at":"2026-09-06T16:16:08.193Z","load_balancing_strategy":"primary","db_duration_s":0.001044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.339660] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"df08b0b89b0afbf828258b08","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-df08b0b89b0afbf828258b08: start","job_status":"start","queue_duration_s":1.575446,"scheduling_latency_s":1.575459} gitlab # [ 180.343893] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"df08b0b89b0afbf828258b08","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-df08b0b89b0afbf828258b08: done: 0.011143 sec","job_status":"done","queue_duration_s":1.575446,"scheduling_latency_s":1.575459,"redis_calls":10,"redis_duration_s":0.001587,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011143,"completed_at":"2026-09-06T16:16:08.205Z","load_balancing_strategy":"primary","db_duration_s":0.001076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.392816] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"079a35adb6762f9dde619965","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-079a35adb6762f9dde619965: start","job_status":"start","queue_duration_s":1.587658,"scheduling_latency_s":1.587673} gitlab # [ 180.393883] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"079a35adb6762f9dde619965","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-079a35adb6762f9dde619965: done: 0.012143 sec","job_status":"done","queue_duration_s":1.587658,"scheduling_latency_s":1.587673,"redis_calls":10,"redis_duration_s":0.002055,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012143,"completed_at":"2026-09-06T16:16:08.218Z","load_balancing_strategy":"primary","db_duration_s":0.001485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.440109] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"42cd796ad69e928b60a5035d","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-42cd796ad69e928b60a5035d: start","job_status":"start","queue_duration_s":1.600915,"scheduling_latency_s":1.60093} gitlab # [ 180.440348] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"42cd796ad69e928b60a5035d","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-42cd796ad69e928b60a5035d: done: 0.012538 sec","job_status":"done","queue_duration_s":1.600915,"scheduling_latency_s":1.60093,"redis_calls":10,"redis_duration_s":0.0017920000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012538,"completed_at":"2026-09-06T16:16:08.232Z","load_balancing_strategy":"primary","db_duration_s":0.002048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.488264] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1cfec9b83459363b1dc6a7fa","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1cfec9b83459363b1dc6a7fa: start","job_status":"start","queue_duration_s":1.61434,"scheduling_latency_s":1.614354} gitlab # [ 180.489055] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1cfec9b83459363b1dc6a7fa","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1cfec9b83459363b1dc6a7fa: done: 0.01413 sec","job_status":"done","queue_duration_s":1.61434,"scheduling_latency_s":1.614354,"redis_calls":10,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000843,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01413,"completed_at":"2026-09-06T16:16:08.247Z","load_balancing_strategy":"primary","db_duration_s":0.004707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.542649] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ae02d3f43853406add6daed3","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ae02d3f43853406add6daed3: start","job_status":"start","queue_duration_s":1.629452,"scheduling_latency_s":1.629466} gitlab # [ 180.543523] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ae02d3f43853406add6daed3","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ae02d3f43853406add6daed3: done: 0.011227 sec","job_status":"done","queue_duration_s":1.629452,"scheduling_latency_s":1.629466,"redis_calls":10,"redis_duration_s":0.001505,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011227,"completed_at":"2026-09-06T16:16:08.259Z","load_balancing_strategy":"primary","db_duration_s":0.001079,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.566760] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"43dd0173c74f65b412be30fc","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-43dd0173c74f65b412be30fc: start","job_status":"start","queue_duration_s":1.641603,"scheduling_latency_s":1.641617} gitlab # [ 180.577973] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"43dd0173c74f65b412be30fc","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-43dd0173c74f65b412be30fc: done: 0.01234 sec","job_status":"done","queue_duration_s":1.641603,"scheduling_latency_s":1.641617,"redis_calls":10,"redis_duration_s":0.001963,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01234,"completed_at":"2026-09-06T16:16:08.272Z","load_balancing_strategy":"primary","db_duration_s":0.001656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.626049] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"067c45619ec01ed2b793a5a8","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-067c45619ec01ed2b793a5a8: start","job_status":"start","queue_duration_s":1.65499,"scheduling_latency_s":1.655004} gitlab # [ 180.626840] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"067c45619ec01ed2b793a5a8","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-067c45619ec01ed2b793a5a8: done: 0.012118 sec","job_status":"done","queue_duration_s":1.65499,"scheduling_latency_s":1.655004,"redis_calls":10,"redis_duration_s":0.002028,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001246,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012118,"completed_at":"2026-09-06T16:16:08.286Z","load_balancing_strategy":"primary","db_duration_s":0.001441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.684273] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2edd6b39fbb2c2f873303768","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2edd6b39fbb2c2f873303768: start","job_status":"start","queue_duration_s":1.668098,"scheduling_latency_s":1.668112} gitlab # [ 180.685030] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2edd6b39fbb2c2f873303768","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2edd6b39fbb2c2f873303768: done: 0.017456 sec","job_status":"done","queue_duration_s":1.668098,"scheduling_latency_s":1.668112,"redis_calls":10,"redis_duration_s":0.001398,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017456,"completed_at":"2026-09-06T16:16:08.304Z","load_balancing_strategy":"primary","db_duration_s":0.011405,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.724171] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9b12d5825b145309c459f053","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9b12d5825b145309c459f053: start","job_status":"start","queue_duration_s":1.68648,"scheduling_latency_s":1.686495} gitlab # [ 180.724352] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9b12d5825b145309c459f053","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9b12d5825b145309c459f053: done: 0.013967 sec","job_status":"done","queue_duration_s":1.68648,"scheduling_latency_s":1.686495,"redis_calls":10,"redis_duration_s":0.001549,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013967,"completed_at":"2026-09-06T16:16:08.319Z","load_balancing_strategy":"primary","db_duration_s":0.004063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.777880] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f71287579d981a2a0d85dfc","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f71287579d981a2a0d85dfc: start","job_status":"start","queue_duration_s":1.701471,"scheduling_latency_s":1.701485} gitlab # [ 180.778683] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f71287579d981a2a0d85dfc","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f71287579d981a2a0d85dfc: done: 0.011284 sec","job_status":"done","queue_duration_s":1.701471,"scheduling_latency_s":1.701485,"redis_calls":10,"redis_duration_s":0.001539,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011284,"completed_at":"2026-09-06T16:16:08.331Z","load_balancing_strategy":"primary","db_duration_s":0.001107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.794843] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ae71c48893fa96b7fb9641ef","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ae71c48893fa96b7fb9641ef: start","job_status":"start","queue_duration_s":1.713704,"scheduling_latency_s":1.713718} gitlab # [ 180.834108] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ae71c48893fa96b7fb9641ef","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ae71c48893fa96b7fb9641ef: done: 0.011237 sec","job_status":"done","queue_duration_s":1.713704,"scheduling_latency_s":1.713718,"redis_calls":10,"redis_duration_s":0.00141,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011237,"completed_at":"2026-09-06T16:16:08.343Z","load_balancing_strategy":"primary","db_duration_s":0.001069,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.878155] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"75efbea17833034b8dd78354","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-75efbea17833034b8dd78354: start","job_status":"start","queue_duration_s":1.725834,"scheduling_latency_s":1.725872} gitlab # [ 180.878849] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"75efbea17833034b8dd78354","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-75efbea17833034b8dd78354: done: 0.011218 sec","job_status":"done","queue_duration_s":1.725834,"scheduling_latency_s":1.725872,"redis_calls":10,"redis_duration_s":0.001443,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011218,"completed_at":"2026-09-06T16:16:08.356Z","load_balancing_strategy":"primary","db_duration_s":0.001079,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.885794] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7a326c741f7ed98b0912db88","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7a326c741f7ed98b0912db88: start","job_status":"start","queue_duration_s":1.737962,"scheduling_latency_s":1.737976} gitlab # [ 180.896937] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7a326c741f7ed98b0912db88","created_at":"2026-09-06T16:16:06.398Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-c9cf14b25f61448b","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"65230ae177b3f9e6ede47f9cef4fdeaa","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.619Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7a326c741f7ed98b0912db88: done: 0.014869 sec","job_status":"done","queue_duration_s":1.737962,"scheduling_latency_s":1.737976,"redis_calls":10,"redis_duration_s":0.001528,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014869,"completed_at":"2026-09-06T16:16:08.371Z","load_balancing_strategy":"primary","db_duration_s":0.005872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.939706] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"292efb484b54e6bd86822cb9","created_at":"2026-09-06T16:16:06.618Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb6fe5114fe127a60cdb6c8e16def663","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.625Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-292efb484b54e6bd86822cb9: start","job_status":"start","queue_duration_s":1.747909,"scheduling_latency_s":1.747924} gitlab # [ 180.943207] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.380Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 180.982125] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"292efb484b54e6bd86822cb9","created_at":"2026-09-06T16:16:06.618Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb6fe5114fe127a60cdb6c8e16def663","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.625Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-292efb484b54e6bd86822cb9: done: 0.014229 sec","job_status":"done","queue_duration_s":1.747909,"scheduling_latency_s":1.747924,"redis_calls":7,"redis_duration_s":0.001377,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000392,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014229,"completed_at":"2026-09-06T16:16:08.387Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.983094] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"c98573d303467f314f6a7fcd","created_at":"2026-09-06T16:16:06.640Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4286241f96f9a9d8eccf43f3f1b3c709","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.654Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-c98573d303467f314f6a7fcd: start","job_status":"start","queue_duration_s":1.734176,"scheduling_latency_s":1.734199} gitlab # [ 181.023379] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.433Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"c98573d303467f314f6a7fcd","created_at":"2026-09-06T16:16:06.640Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4286241f96f9a9d8eccf43f3f1b3c709","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.654Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-c98573d303467f314f6a7fcd: done: 0.044784 sec","job_status":"done","queue_duration_s":1.734176,"scheduling_latency_s":1.734199,"redis_calls":12,"redis_duration_s":0.001994,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000358,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000713,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000447543,"exclusive_lock_hold_duration_s":0.0039359749999999995,"duration_s":0.044784,"completed_at":"2026-09-06T16:16:08.433Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.065520] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fea8a08e0618107724d7f677","created_at":"2026-09-06T16:16:06.667Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22cddeeaed9a9164fecc5bf4bfc393c2","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.671Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fea8a08e0618107724d7f677: start","job_status":"start","queue_duration_s":1.762946,"scheduling_latency_s":1.762961} gitlab # [ 181.066108] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fea8a08e0618107724d7f677","created_at":"2026-09-06T16:16:06.667Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22cddeeaed9a9164fecc5bf4bfc393c2","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.671Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fea8a08e0618107724d7f677: done: 0.23185 sec","job_status":"done","queue_duration_s":1.762946,"scheduling_latency_s":1.762961,"redis_calls":562,"redis_duration_s":0.077724,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.076958,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.180645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.23185,"completed_at":"2026-09-06T16:16:08.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.113299] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.668Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c54252ae80a76d5d6bd9df30","created_at":"2026-09-06T16:16:06.678Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5922b8b40ef8698ebeef4b5f291b74af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.682Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c54252ae80a76d5d6bd9df30: start","job_status":"start","queue_duration_s":1.985148,"scheduling_latency_s":1.98517} gitlab # [ 181.114045] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c54252ae80a76d5d6bd9df30","created_at":"2026-09-06T16:16:06.678Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5922b8b40ef8698ebeef4b5f291b74af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.682Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c54252ae80a76d5d6bd9df30: done: 0.14851 sec","job_status":"done","queue_duration_s":1.985148,"scheduling_latency_s":1.98517,"redis_calls":11,"redis_duration_s":0.002044,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00032127,"exclusive_lock_hold_duration_s":0.133433719,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.14851,"completed_at":"2026-09-06T16:16:08.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.151665] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2e2846b31c481e9d9e99bacd","created_at":"2026-09-06T16:16:06.698Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1a009cc50ce5626d26e1794d069530b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.702Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2e2846b31c481e9d9e99bacd: start","job_status":"start","queue_duration_s":2.115392,"scheduling_latency_s":2.115407} gitlab # [ 181.152611] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2e2846b31c481e9d9e99bacd","created_at":"2026-09-06T16:16:06.698Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1a009cc50ce5626d26e1794d069530b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.702Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2e2846b31c481e9d9e99bacd: done: 0.017539 sec","job_status":"done","queue_duration_s":2.115392,"scheduling_latency_s":2.115407,"redis_calls":7,"redis_duration_s":0.001425,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.017539,"completed_at":"2026-09-06T16:16:08.835Z","load_balancing_strategy":"primary","db_duration_s":0.006877,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.193896] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4a9e7876c309e449eed51c19","created_at":"2026-09-06T16:16:06.712Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7155adf75d2cedc0e14c23639c34fd63","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.714Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4a9e7876c309e449eed51c19: start","job_status":"start","queue_duration_s":2.12163,"scheduling_latency_s":2.121646} gitlab # [ 181.194915] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4a9e7876c309e449eed51c19","created_at":"2026-09-06T16:16:06.712Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7155adf75d2cedc0e14c23639c34fd63","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.714Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4a9e7876c309e449eed51c19: done: 0.013193 sec","job_status":"done","queue_duration_s":2.12163,"scheduling_latency_s":2.121646,"redis_calls":9,"redis_duration_s":0.001337,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000324343,"exclusive_lock_hold_duration_s":0.004474591,"duration_s":0.013193,"completed_at":"2026-09-06T16:16:08.849Z","load_balancing_strategy":"primary","db_duration_s":0.003088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.254243] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.850Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"319f125c7604c5015dbb8cd7","created_at":"2026-09-06T16:16:06.724Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82c12acba47ebce7d72b41f1a40b2308","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.726Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-319f125c7604c5015dbb8cd7: start","job_status":"start","queue_duration_s":2.123879,"scheduling_latency_s":2.123894} gitlab # [ 181.255236] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.855Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 181.256203] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"319f125c7604c5015dbb8cd7","created_at":"2026-09-06T16:16:06.724Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82c12acba47ebce7d72b41f1a40b2308","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.726Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-319f125c7604c5015dbb8cd7: done: 0.00815 sec","job_status":"done","queue_duration_s":2.123879,"scheduling_latency_s":2.123894,"redis_calls":7,"redis_duration_s":0.000968,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.00815,"completed_at":"2026-09-06T16:16:08.858Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.307899] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0dfdeaf2a786304ce8b97c93","created_at":"2026-09-06T16:16:06.731Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a14aa2da65bb2e79f3edb4652902e97e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:16:06.734Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0dfdeaf2a786304ce8b97c93: start","job_status":"start","queue_duration_s":2.124728,"scheduling_latency_s":2.124742} gitlab # [ 181.308551] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.864Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 181.309326] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0dfdeaf2a786304ce8b97c93","created_at":"2026-09-06T16:16:06.731Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-249976c0c4474370","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a14aa2da65bb2e79f3edb4652902e97e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:16:06.734Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0dfdeaf2a786304ce8b97c93: done: 0.007898 sec","job_status":"done","queue_duration_s":2.124728,"scheduling_latency_s":2.124742,"redis_calls":7,"redis_duration_s":0.0010170000000000001,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007898,"completed_at":"2026-09-06T16:16:08.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.349542] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.868Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c965b0ed833c4bb7476afb16","created_at":"2026-09-06T16:16:06.747Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-9aa4bc3f0d494679","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.754Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c965b0ed833c4bb7476afb16: start","job_status":"start","queue_duration_s":2.114097,"scheduling_latency_s":2.114109} gitlab # [ 181.350338] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.875Z","message":"No existing merge request to be cleaned up."} gitlab # [ 181.351200] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.880Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c965b0ed833c4bb7476afb16","created_at":"2026-09-06T16:16:06.747Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-9aa4bc3f0d494679","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.754Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c965b0ed833c4bb7476afb16: done: 0.012483 sec","job_status":"done","queue_duration_s":2.114097,"scheduling_latency_s":2.114109,"redis_calls":10,"redis_duration_s":0.001451,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012483,"completed_at":"2026-09-06T16:16:08.880Z","load_balancing_strategy":"primary","db_duration_s":0.001353,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.398355] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.881Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8ecb7842fb775b1273f783c","created_at":"2026-09-06T16:16:06.747Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-9aa4bc3f0d494679","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.754Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8ecb7842fb775b1273f783c: start","job_status":"start","queue_duration_s":2.127509,"scheduling_latency_s":2.127523} gitlab # [ 181.399282] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.888Z","message":"No existing merge request to be cleaned up."} gitlab # [ 181.400158] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.893Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8ecb7842fb775b1273f783c","created_at":"2026-09-06T16:16:06.747Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-9aa4bc3f0d494679","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.754Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8ecb7842fb775b1273f783c: done: 0.011625 sec","job_status":"done","queue_duration_s":2.127509,"scheduling_latency_s":2.127523,"redis_calls":10,"redis_duration_s":0.0015019999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000866,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011625,"completed_at":"2026-09-06T16:16:08.893Z","load_balancing_strategy":"primary","db_duration_s":0.001202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.440332] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.894Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e8c78b70d5fa9c323ce74c26","created_at":"2026-09-06T16:16:06.747Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-9aa4bc3f0d494679","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.754Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e8c78b70d5fa9c323ce74c26: start","job_status":"start","queue_duration_s":2.140183,"scheduling_latency_s":2.140197} gitlab # [ 181.441250] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.901Z","message":"No existing merge request to be cleaned up."} gitlab # [ 181.442167] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.905Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e8c78b70d5fa9c323ce74c26","created_at":"2026-09-06T16:16:06.747Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-9aa4bc3f0d494679","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.754Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e8c78b70d5fa9c323ce74c26: done: 0.011492 sec","job_status":"done","queue_duration_s":2.140183,"scheduling_latency_s":2.140197,"redis_calls":10,"redis_duration_s":0.0014190000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000849,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011492,"completed_at":"2026-09-06T16:16:08.905Z","load_balancing_strategy":"primary","db_duration_s":0.001048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.484803] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.906Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac09613e1aeea92f63368d03","created_at":"2026-09-06T16:16:06.747Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-9aa4bc3f0d494679","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.754Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac09613e1aeea92f63368d03: start","job_status":"start","queue_duration_s":2.152616,"scheduling_latency_s":2.15263} gitlab # [ 181.485566] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.913Z","message":"No existing merge request to be cleaned up."} gitlab # [ 181.486306] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.918Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac09613e1aeea92f63368d03","created_at":"2026-09-06T16:16:06.747Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-9aa4bc3f0d494679","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"da85bab535d0889e10f9d73b26382b3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.754Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac09613e1aeea92f63368d03: done: 0.011437 sec","job_status":"done","queue_duration_s":2.152616,"scheduling_latency_s":2.15263,"redis_calls":10,"redis_duration_s":0.001627,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011437,"completed_at":"2026-09-06T16:16:08.918Z","load_balancing_strategy":"primary","db_duration_s":0.001019,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.535210] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed16f208b345b5720fdd0879","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed16f208b345b5720fdd0879: start","job_status":"start","queue_duration_s":2.059774,"scheduling_latency_s":2.059788} gitlab # [ 181.536154] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed16f208b345b5720fdd0879","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed16f208b345b5720fdd0879: done: 0.00974 sec","job_status":"done","queue_duration_s":2.059774,"scheduling_latency_s":2.059788,"redis_calls":13,"redis_duration_s":0.00176,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00974,"completed_at":"2026-09-06T16:16:08.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.579842] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"791e424364663419977294ef","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-791e424364663419977294ef: start","job_status":"start","queue_duration_s":2.070471,"scheduling_latency_s":2.070483} gitlab # [ 181.580820] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"791e424364663419977294ef","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-791e424364663419977294ef: done: 0.009648 sec","job_status":"done","queue_duration_s":2.070471,"scheduling_latency_s":2.070483,"redis_calls":13,"redis_duration_s":0.001791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001259,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009648,"completed_at":"2026-09-06T16:16:08.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.641142] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bdb1ccc7e3d15fafc642a36c","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bdb1ccc7e3d15fafc642a36c: start","job_status":"start","queue_duration_s":2.081108,"scheduling_latency_s":2.081122} gitlab # [ 181.642127] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bdb1ccc7e3d15fafc642a36c","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bdb1ccc7e3d15fafc642a36c: done: 0.009756 sec","job_status":"done","queue_duration_s":2.081108,"scheduling_latency_s":2.081122,"redis_calls":13,"redis_duration_s":0.002098,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001392,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009756,"completed_at":"2026-09-06T16:16:08.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.684660] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a859dc06b25753786717146d","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a859dc06b25753786717146d: start","job_status":"start","queue_duration_s":2.091971,"scheduling_latency_s":2.091984} gitlab # [ 181.685717] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a859dc06b25753786717146d","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a859dc06b25753786717146d: done: 0.010247 sec","job_status":"done","queue_duration_s":2.091971,"scheduling_latency_s":2.091984,"redis_calls":13,"redis_duration_s":0.0022900000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001561,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010247,"completed_at":"2026-09-06T16:16:08.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.738633] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"790bc913655fbe8b0c04ecd3","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-790bc913655fbe8b0c04ecd3: start","job_status":"start","queue_duration_s":2.103303,"scheduling_latency_s":2.103318} gitlab # [ 181.739489] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"790bc913655fbe8b0c04ecd3","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-790bc913655fbe8b0c04ecd3: done: 0.010282 sec","job_status":"done","queue_duration_s":2.103303,"scheduling_latency_s":2.103318,"redis_calls":13,"redis_duration_s":0.002399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010282,"completed_at":"2026-09-06T16:16:08.972Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.778322] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ff62a44f8d9496f4e22993c","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ff62a44f8d9496f4e22993c: start","job_status":"start","queue_duration_s":2.114791,"scheduling_latency_s":2.114805} gitlab # [ 181.779157] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ff62a44f8d9496f4e22993c","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ff62a44f8d9496f4e22993c: done: 0.010077 sec","job_status":"done","queue_duration_s":2.114791,"scheduling_latency_s":2.114805,"redis_calls":13,"redis_duration_s":0.002141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000708,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001433,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010077,"completed_at":"2026-09-06T16:16:08.984Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.818641] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9011e28b4c22960f048587e","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9011e28b4c22960f048587e: start","job_status":"start","queue_duration_s":2.125812,"scheduling_latency_s":2.125826} gitlab # [ 181.819730] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9011e28b4c22960f048587e","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9011e28b4c22960f048587e: done: 0.009391 sec","job_status":"done","queue_duration_s":2.125812,"scheduling_latency_s":2.125826,"redis_calls":13,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009391,"completed_at":"2026-09-06T16:16:08.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.872863] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:08.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"647d255e1a64959dc812d56f","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-647d255e1a64959dc812d56f: start","job_status":"start","queue_duration_s":2.136108,"scheduling_latency_s":2.136122} gitlab # [ 181.873839] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"647d255e1a64959dc812d56f","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-647d255e1a64959dc812d56f: done: 0.009479 sec","job_status":"done","queue_duration_s":2.136108,"scheduling_latency_s":2.136122,"redis_calls":13,"redis_duration_s":0.0017829999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009479,"completed_at":"2026-09-06T16:16:09.004Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.916247] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25ec50168a028d3ff5677406","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25ec50168a028d3ff5677406: start","job_status":"start","queue_duration_s":2.146578,"scheduling_latency_s":2.146594} gitlab # [ 181.917313] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25ec50168a028d3ff5677406","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25ec50168a028d3ff5677406: done: 0.00951 sec","job_status":"done","queue_duration_s":2.146578,"scheduling_latency_s":2.146594,"redis_calls":13,"redis_duration_s":0.001844,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00073,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00951,"completed_at":"2026-09-06T16:16:09.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.968315] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65fd5674cdc8dbc48fcaae50","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65fd5674cdc8dbc48fcaae50: start","job_status":"start","queue_duration_s":2.156993,"scheduling_latency_s":2.157007} gitlab # [ 181.969158] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65fd5674cdc8dbc48fcaae50","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65fd5674cdc8dbc48fcaae50: done: 0.009427 sec","job_status":"done","queue_duration_s":2.156993,"scheduling_latency_s":2.157007,"redis_calls":13,"redis_duration_s":0.001728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009427,"completed_at":"2026-09-06T16:16:09.025Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.015448] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6229eb8897001971a70f279","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6229eb8897001971a70f279: start","job_status":"start","queue_duration_s":2.167312,"scheduling_latency_s":2.167326} gitlab # [ 182.016327] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6229eb8897001971a70f279","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6229eb8897001971a70f279: done: 0.009361 sec","job_status":"done","queue_duration_s":2.167312,"scheduling_latency_s":2.167326,"redis_calls":13,"redis_duration_s":0.001667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009361,"completed_at":"2026-09-06T16:16:09.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.027883] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1874f43c990cdd0f7748e40","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1874f43c990cdd0f7748e40: start","job_status":"start","queue_duration_s":2.177584,"scheduling_latency_s":2.177598} gitlab # [ 182.076208] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1874f43c990cdd0f7748e40","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1874f43c990cdd0f7748e40: done: 0.009481 sec","job_status":"done","queue_duration_s":2.177584,"scheduling_latency_s":2.177598,"redis_calls":13,"redis_duration_s":0.0017430000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009481,"completed_at":"2026-09-06T16:16:09.046Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.077662] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b18f2bd01a37d96b44588fc8","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b18f2bd01a37d96b44588fc8: start","job_status":"start","queue_duration_s":2.188027,"scheduling_latency_s":2.18804} gitlab # [ 182.085166] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b18f2bd01a37d96b44588fc8","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b18f2bd01a37d96b44588fc8: done: 0.009451 sec","job_status":"done","queue_duration_s":2.188027,"scheduling_latency_s":2.18804,"redis_calls":13,"redis_duration_s":0.001667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009451,"completed_at":"2026-09-06T16:16:09.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.127747] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e72cbf4bda29adda6bc3c42","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e72cbf4bda29adda6bc3c42: start","job_status":"start","queue_duration_s":2.198425,"scheduling_latency_s":2.198439} gitlab # [ 182.130377] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e72cbf4bda29adda6bc3c42","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e72cbf4bda29adda6bc3c42: done: 0.009533 sec","job_status":"done","queue_duration_s":2.198425,"scheduling_latency_s":2.198439,"redis_calls":13,"redis_duration_s":0.001727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009533,"completed_at":"2026-09-06T16:16:09.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.185143] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ceb522d6e48917107fb8bea9","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ceb522d6e48917107fb8bea9: start","job_status":"start","queue_duration_s":2.208891,"scheduling_latency_s":2.208905} gitlab # [ 182.186138] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ceb522d6e48917107fb8bea9","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ceb522d6e48917107fb8bea9: done: 0.009507 sec","job_status":"done","queue_duration_s":2.208891,"scheduling_latency_s":2.208905,"redis_calls":13,"redis_duration_s":0.00176,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009507,"completed_at":"2026-09-06T16:16:09.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.241838] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"220005968cbbdd0f51d4a0c6","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-220005968cbbdd0f51d4a0c6: start","job_status":"start","queue_duration_s":2.219308,"scheduling_latency_s":2.219322} gitlab # [ 182.242849] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"220005968cbbdd0f51d4a0c6","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-220005968cbbdd0f51d4a0c6: done: 0.009553 sec","job_status":"done","queue_duration_s":2.219308,"scheduling_latency_s":2.219322,"redis_calls":13,"redis_duration_s":0.001728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009553,"completed_at":"2026-09-06T16:16:09.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.283874] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7a6d6c50b9ff6273aa8b84f","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7a6d6c50b9ff6273aa8b84f: start","job_status":"start","queue_duration_s":2.229802,"scheduling_latency_s":2.229817} gitlab # [ 182.284866] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7a6d6c50b9ff6273aa8b84f","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7a6d6c50b9ff6273aa8b84f: done: 0.009443 sec","job_status":"done","queue_duration_s":2.229802,"scheduling_latency_s":2.229817,"redis_calls":13,"redis_duration_s":0.001696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009443,"completed_at":"2026-09-06T16:16:09.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.334832] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ced1125952f4882e243ad263","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ced1125952f4882e243ad263: start","job_status":"start","queue_duration_s":2.240152,"scheduling_latency_s":2.240167} gitlab # [ 182.335952] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ced1125952f4882e243ad263","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ced1125952f4882e243ad263: done: 0.009607 sec","job_status":"done","queue_duration_s":2.240152,"scheduling_latency_s":2.240167,"redis_calls":13,"redis_duration_s":0.0017079999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009607,"completed_at":"2026-09-06T16:16:09.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.375471] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"489fbb3c6725c5ff1b2bc6b0","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-489fbb3c6725c5ff1b2bc6b0: start","job_status":"start","queue_duration_s":2.250746,"scheduling_latency_s":2.25076} gitlab # [ 182.376398] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"489fbb3c6725c5ff1b2bc6b0","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-489fbb3c6725c5ff1b2bc6b0: done: 0.009668 sec","job_status":"done","queue_duration_s":2.250746,"scheduling_latency_s":2.25076,"redis_calls":13,"redis_duration_s":0.001967,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001372,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009668,"completed_at":"2026-09-06T16:16:09.119Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.392884] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddfa396d90ef1166d0ebb5b6","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddfa396d90ef1166d0ebb5b6: start","job_status":"start","queue_duration_s":2.261377,"scheduling_latency_s":2.261391} gitlab # [ 182.439755] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddfa396d90ef1166d0ebb5b6","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddfa396d90ef1166d0ebb5b6: done: 0.009506 sec","job_status":"done","queue_duration_s":2.261377,"scheduling_latency_s":2.261391,"redis_calls":13,"redis_duration_s":0.001717,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009506,"completed_at":"2026-09-06T16:16:09.130Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.441334] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10c6c807bbee9bbba31ca489","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10c6c807bbee9bbba31ca489: start","job_status":"start","queue_duration_s":2.271834,"scheduling_latency_s":2.271848} gitlab # [ 182.450946] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10c6c807bbee9bbba31ca489","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10c6c807bbee9bbba31ca489: done: 0.009385 sec","job_status":"done","queue_duration_s":2.271834,"scheduling_latency_s":2.271848,"redis_calls":13,"redis_duration_s":0.001743,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009385,"completed_at":"2026-09-06T16:16:09.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.497976] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4022802a6dae08310d87e9b1","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4022802a6dae08310d87e9b1: start","job_status":"start","queue_duration_s":2.282159,"scheduling_latency_s":2.282173} gitlab # [ 182.499519] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4022802a6dae08310d87e9b1","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4022802a6dae08310d87e9b1: done: 0.013053 sec","job_status":"done","queue_duration_s":2.282159,"scheduling_latency_s":2.282173,"redis_calls":13,"redis_duration_s":0.0052510000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00406,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013053,"completed_at":"2026-09-06T16:16:09.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.562750] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c79558cad97cee9b143ffed","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c79558cad97cee9b143ffed: start","job_status":"start","queue_duration_s":2.296144,"scheduling_latency_s":2.296157} gitlab # [ 182.565190] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c79558cad97cee9b143ffed","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c79558cad97cee9b143ffed: done: 0.009647 sec","job_status":"done","queue_duration_s":2.296144,"scheduling_latency_s":2.296157,"redis_calls":13,"redis_duration_s":0.0017499999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009647,"completed_at":"2026-09-06T16:16:09.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.582849] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dad6de5d15ba7cec0f84e4e","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dad6de5d15ba7cec0f84e4e: start","job_status":"start","queue_duration_s":2.306701,"scheduling_latency_s":2.306715} gitlab # [ 182.593728] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dad6de5d15ba7cec0f84e4e","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dad6de5d15ba7cec0f84e4e: done: 0.009497 sec","job_status":"done","queue_duration_s":2.306701,"scheduling_latency_s":2.306715,"redis_calls":13,"redis_duration_s":0.00185,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009497,"completed_at":"2026-09-06T16:16:09.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.652258] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7621ccfe7938e2d2a1d7b8bd","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7621ccfe7938e2d2a1d7b8bd: start","job_status":"start","queue_duration_s":2.317081,"scheduling_latency_s":2.317095} gitlab # [ 182.652929] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7621ccfe7938e2d2a1d7b8bd","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7621ccfe7938e2d2a1d7b8bd: done: 0.010157 sec","job_status":"done","queue_duration_s":2.317081,"scheduling_latency_s":2.317095,"redis_calls":13,"redis_duration_s":0.002263,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001548,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010157,"completed_at":"2026-09-06T16:16:09.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.675939] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19b56e97f2f6268f4cbfb887","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19b56e97f2f6268f4cbfb887: start","job_status":"start","queue_duration_s":2.328322,"scheduling_latency_s":2.328336} gitlab # [ 182.718710] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19b56e97f2f6268f4cbfb887","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19b56e97f2f6268f4cbfb887: done: 0.010321 sec","job_status":"done","queue_duration_s":2.328322,"scheduling_latency_s":2.328336,"redis_calls":13,"redis_duration_s":0.00236,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001698,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010321,"completed_at":"2026-09-06T16:16:09.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.760174] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"805e1da3590f447ef4c38faf","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-805e1da3590f447ef4c38faf: start","job_status":"start","queue_duration_s":2.339631,"scheduling_latency_s":2.339646} gitlab # [ 182.761126] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"805e1da3590f447ef4c38faf","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-805e1da3590f447ef4c38faf: done: 0.009412 sec","job_status":"done","queue_duration_s":2.339631,"scheduling_latency_s":2.339646,"redis_calls":13,"redis_duration_s":0.001753,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009412,"completed_at":"2026-09-06T16:16:09.208Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.771902] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c341d1af97ca532c9e8812e7","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c341d1af97ca532c9e8812e7: start","job_status":"start","queue_duration_s":2.350025,"scheduling_latency_s":2.350039} gitlab # [ 182.815771] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c341d1af97ca532c9e8812e7","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c341d1af97ca532c9e8812e7: done: 0.009609 sec","job_status":"done","queue_duration_s":2.350025,"scheduling_latency_s":2.350039,"redis_calls":13,"redis_duration_s":0.0019230000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001378,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009609,"completed_at":"2026-09-06T16:16:09.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.821571] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd662ca24aaa4e0fe6d8c8fd","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd662ca24aaa4e0fe6d8c8fd: start","job_status":"start","queue_duration_s":2.360562,"scheduling_latency_s":2.360575} gitlab # [ 182.865161] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd662ca24aaa4e0fe6d8c8fd","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd662ca24aaa4e0fe6d8c8fd: done: 0.009359 sec","job_status":"done","queue_duration_s":2.360562,"scheduling_latency_s":2.360575,"redis_calls":13,"redis_duration_s":0.00168,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009359,"completed_at":"2026-09-06T16:16:09.229Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.871591] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fd294f184e10de8bff33808","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fd294f184e10de8bff33808: start","job_status":"start","queue_duration_s":2.370801,"scheduling_latency_s":2.370815} gitlab # [ 182.917921] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fd294f184e10de8bff33808","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fd294f184e10de8bff33808: done: 0.009614 sec","job_status":"done","queue_duration_s":2.370801,"scheduling_latency_s":2.370815,"redis_calls":13,"redis_duration_s":0.001827,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009614,"completed_at":"2026-09-06T16:16:09.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.920081] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b643b74cf705d551cbc23d10","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b643b74cf705d551cbc23d10: start","job_status":"start","queue_duration_s":2.381374,"scheduling_latency_s":2.381386} gitlab # [ 182.968750] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b643b74cf705d551cbc23d10","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b643b74cf705d551cbc23d10: done: 0.009526 sec","job_status":"done","queue_duration_s":2.381374,"scheduling_latency_s":2.381386,"redis_calls":13,"redis_duration_s":0.001781,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009526,"completed_at":"2026-09-06T16:16:09.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.970049] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b945194e8424f059f8940e5b","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b945194e8424f059f8940e5b: start","job_status":"start","queue_duration_s":2.391862,"scheduling_latency_s":2.391877} gitlab # [ 182.981632] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b945194e8424f059f8940e5b","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b945194e8424f059f8940e5b: done: 0.009399 sec","job_status":"done","queue_duration_s":2.391862,"scheduling_latency_s":2.391877,"redis_calls":13,"redis_duration_s":0.00173,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009399,"completed_at":"2026-09-06T16:16:09.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.041085] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6299b40ed9e71ed10fc4e8ef","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6299b40ed9e71ed10fc4e8ef: start","job_status":"start","queue_duration_s":2.402202,"scheduling_latency_s":2.402216} gitlab # [ 183.041809] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6299b40ed9e71ed10fc4e8ef","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6299b40ed9e71ed10fc4e8ef: done: 0.009557 sec","job_status":"done","queue_duration_s":2.402202,"scheduling_latency_s":2.402216,"redis_calls":13,"redis_duration_s":0.001731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009557,"completed_at":"2026-09-06T16:16:09.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.092115] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a167a8bb0561b331a3bed16b","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a167a8bb0561b331a3bed16b: start","job_status":"start","queue_duration_s":2.412713,"scheduling_latency_s":2.412727} gitlab # [ 183.092859] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a167a8bb0561b331a3bed16b","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a167a8bb0561b331a3bed16b: done: 0.013957 sec","job_status":"done","queue_duration_s":2.412713,"scheduling_latency_s":2.412727,"redis_calls":13,"redis_duration_s":0.002112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013957,"completed_at":"2026-09-06T16:16:09.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.140164] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"765db393f66c3712eb34cd04","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-765db393f66c3712eb34cd04: start","job_status":"start","queue_duration_s":2.427675,"scheduling_latency_s":2.427689} gitlab # [ 183.141075] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"765db393f66c3712eb34cd04","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-765db393f66c3712eb34cd04: done: 0.009759 sec","job_status":"done","queue_duration_s":2.427675,"scheduling_latency_s":2.427689,"redis_calls":13,"redis_duration_s":0.0017530000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009759,"completed_at":"2026-09-06T16:16:09.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.186846] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"034ab9a92168b65d2fa870e9","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-034ab9a92168b65d2fa870e9: start","job_status":"start","queue_duration_s":2.43856,"scheduling_latency_s":2.438576} gitlab # [ 183.187663] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"034ab9a92168b65d2fa870e9","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-034ab9a92168b65d2fa870e9: done: 0.011219 sec","job_status":"done","queue_duration_s":2.43856,"scheduling_latency_s":2.438576,"redis_calls":13,"redis_duration_s":0.001946,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00074,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011219,"completed_at":"2026-09-06T16:16:09.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.241857] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc09a477fe75e1ce48c4bfc9","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc09a477fe75e1ce48c4bfc9: start","job_status":"start","queue_duration_s":2.450688,"scheduling_latency_s":2.450701} gitlab # [ 183.242739] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc09a477fe75e1ce48c4bfc9","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc09a477fe75e1ce48c4bfc9: done: 0.0103 sec","job_status":"done","queue_duration_s":2.450688,"scheduling_latency_s":2.450701,"redis_calls":13,"redis_duration_s":0.001988,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0103,"completed_at":"2026-09-06T16:16:09.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.300755] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"866e234234a8eb324e438f5c","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-866e234234a8eb324e438f5c: start","job_status":"start","queue_duration_s":2.461881,"scheduling_latency_s":2.461894} gitlab # [ 183.301591] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"866e234234a8eb324e438f5c","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-866e234234a8eb324e438f5c: done: 0.010061 sec","job_status":"done","queue_duration_s":2.461881,"scheduling_latency_s":2.461894,"redis_calls":13,"redis_duration_s":0.001693,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010061,"completed_at":"2026-09-06T16:16:09.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.344790] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"012d69dad8332177e457197a","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-012d69dad8332177e457197a: start","job_status":"start","queue_duration_s":2.472905,"scheduling_latency_s":2.472919} gitlab # [ 183.345856] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"012d69dad8332177e457197a","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-012d69dad8332177e457197a: done: 0.009992 sec","job_status":"done","queue_duration_s":2.472905,"scheduling_latency_s":2.472919,"redis_calls":13,"redis_duration_s":0.001788,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009992,"completed_at":"2026-09-06T16:16:09.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.393577] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddaf9158671e4e54d1bf1925","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddaf9158671e4e54d1bf1925: start","job_status":"start","queue_duration_s":2.483822,"scheduling_latency_s":2.483836} gitlab # [ 183.394500] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddaf9158671e4e54d1bf1925","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddaf9158671e4e54d1bf1925: done: 0.009933 sec","job_status":"done","queue_duration_s":2.483822,"scheduling_latency_s":2.483836,"redis_calls":13,"redis_duration_s":0.001745,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009933,"completed_at":"2026-09-06T16:16:09.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.443787] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a7e8a58487bc45c6f2c85c0","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a7e8a58487bc45c6f2c85c0: start","job_status":"start","queue_duration_s":2.494667,"scheduling_latency_s":2.494681} gitlab # [ 183.444608] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a7e8a58487bc45c6f2c85c0","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a7e8a58487bc45c6f2c85c0: done: 0.009966 sec","job_status":"done","queue_duration_s":2.494667,"scheduling_latency_s":2.494681,"redis_calls":13,"redis_duration_s":0.0016979999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009966,"completed_at":"2026-09-06T16:16:09.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.491581] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"027a9bc4d202a4b0fb708ff1","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-027a9bc4d202a4b0fb708ff1: start","job_status":"start","queue_duration_s":2.505564,"scheduling_latency_s":2.505579} gitlab # [ 183.492630] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"027a9bc4d202a4b0fb708ff1","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-027a9bc4d202a4b0fb708ff1: done: 0.010607 sec","job_status":"done","queue_duration_s":2.505564,"scheduling_latency_s":2.505579,"redis_calls":13,"redis_duration_s":0.002437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010607,"completed_at":"2026-09-06T16:16:09.375Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.508683] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd274d1ae17e35adf2b10875","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd274d1ae17e35adf2b10875: start","job_status":"start","queue_duration_s":2.517195,"scheduling_latency_s":2.517209} gitlab # [ 183.551062] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd274d1ae17e35adf2b10875","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd274d1ae17e35adf2b10875: done: 0.009938 sec","job_status":"done","queue_duration_s":2.517195,"scheduling_latency_s":2.517209,"redis_calls":13,"redis_duration_s":0.001589,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009938,"completed_at":"2026-09-06T16:16:09.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.553660] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af7c7207e7be5a54b006ea6f","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af7c7207e7be5a54b006ea6f: start","job_status":"start","queue_duration_s":2.528237,"scheduling_latency_s":2.52825} gitlab # [ 183.564832] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af7c7207e7be5a54b006ea6f","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af7c7207e7be5a54b006ea6f: done: 0.009957 sec","job_status":"done","queue_duration_s":2.528237,"scheduling_latency_s":2.52825,"redis_calls":13,"redis_duration_s":0.001741,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009957,"completed_at":"2026-09-06T16:16:09.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.606150] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"846ec717a1dc738a8c7e32f3","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-846ec717a1dc738a8c7e32f3: start","job_status":"start","queue_duration_s":2.539256,"scheduling_latency_s":2.53927} gitlab # [ 183.646976] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"846ec717a1dc738a8c7e32f3","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-846ec717a1dc738a8c7e32f3: done: 0.010025 sec","job_status":"done","queue_duration_s":2.539256,"scheduling_latency_s":2.53927,"redis_calls":13,"redis_duration_s":0.0017569999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010025,"completed_at":"2026-09-06T16:16:09.408Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.647958] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81725d4fc14feb6ff67836e2","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81725d4fc14feb6ff67836e2: start","job_status":"start","queue_duration_s":2.550394,"scheduling_latency_s":2.550407} gitlab # [ 183.655569] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81725d4fc14feb6ff67836e2","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81725d4fc14feb6ff67836e2: done: 0.010337 sec","job_status":"done","queue_duration_s":2.550394,"scheduling_latency_s":2.550407,"redis_calls":13,"redis_duration_s":0.001873,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001287,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010337,"completed_at":"2026-09-06T16:16:09.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.699833] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47d06bbf2fa5a0978cb46be9","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47d06bbf2fa5a0978cb46be9: start","job_status":"start","queue_duration_s":2.561793,"scheduling_latency_s":2.561807} gitlab # [ 183.701521] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47d06bbf2fa5a0978cb46be9","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47d06bbf2fa5a0978cb46be9: done: 0.009873 sec","job_status":"done","queue_duration_s":2.561793,"scheduling_latency_s":2.561807,"redis_calls":13,"redis_duration_s":0.00164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009873,"completed_at":"2026-09-06T16:16:09.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.754839] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"809a0e1cac598c1286b75ed7","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-809a0e1cac598c1286b75ed7: start","job_status":"start","queue_duration_s":2.572588,"scheduling_latency_s":2.572603} gitlab # [ 183.755901] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"809a0e1cac598c1286b75ed7","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-809a0e1cac598c1286b75ed7: done: 0.010091 sec","job_status":"done","queue_duration_s":2.572588,"scheduling_latency_s":2.572603,"redis_calls":13,"redis_duration_s":0.001914,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001326,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010091,"completed_at":"2026-09-06T16:16:09.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.809703] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e7813d9615ec63ea2518e5","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e7813d9615ec63ea2518e5: start","job_status":"start","queue_duration_s":2.583568,"scheduling_latency_s":2.583582} gitlab # [ 183.810743] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e7813d9615ec63ea2518e5","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e7813d9615ec63ea2518e5: done: 0.009891 sec","job_status":"done","queue_duration_s":2.583568,"scheduling_latency_s":2.583582,"redis_calls":13,"redis_duration_s":0.0018319999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009891,"completed_at":"2026-09-06T16:16:09.452Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.827911] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4126be04b7cb678e674f4968","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4126be04b7cb678e674f4968: start","job_status":"start","queue_duration_s":2.594316,"scheduling_latency_s":2.594329} gitlab # [ 183.872406] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:09.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4126be04b7cb678e674f4968","created_at":"2026-09-06T16:16:06.765Z","trace_propagation_headers":{"sentry-trace":"e37bb724cbf94e68809becb342d6942f-3404bb0dcee5494a","baggage":"sentry-trace_id=e37bb724cbf94e68809becb342d6942f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4511a15d18a08f3898b26581c7ab0df9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:06.859Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4126be04b7cb678e674f4968: done: 0.010055 sec","job_status":"done","queue_duration_s":2.594316,"scheduling_latency_s":2.594329,"redis_calls":13,"redis_duration_s":0.002037,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000679,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001358,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010055,"completed_at":"2026-09-06T16:16:09.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.057582] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 4453972012 wd_nsec: 4453971949 gitlab # [ 188.977105] postgres[2461]: [2461] ERROR: relation "table_without_models" does not exist gitlab # [ 188.979612] postgres[2461]: [2461] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 191.905785] runuser[2244]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 191.908763] setup-gitlab-start[2227]: + systemctl restart gitlab-sidekiq.service gitlab # [ 191.933474] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 191.935864] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:20.969Z","message":"Shutting down"} gitlab # [ 191.936934] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:20.969Z","message":"Scheduler exiting..."} gitlab # [ 191.941915] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:20.975Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 191.942604] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:20.976Z","message":"Scheduler exiting..."} gitlab # [ 192.449808] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:21.483Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 195.955460] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:24.988Z","message":"Bye!"} gitlab # [ 195.980922] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:16:25.014Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1640,"worker_id":"sidekiq_0","memwd_rss_bytes":950177792,"retry":0} gitlab # [ 197.110114] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 197.113105] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 197.115124] systemd[1]: gitlab-sidekiq.service: Consumed 1min 585ms CPU time over 2min 1.178s wall clock time, 968.5M memory peak, 5.3M read from disk. gitlab # [ 197.125767] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 197.132624] setup-gitlab-start[2227]: + mkdir -p /var/lib/secrets gitlab # [ 197.137783] setup-gitlab-start[2227]: + echo nixbot-test-token-of-20-chars gitlab # [ 197.139561] setup-gitlab-start[2227]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 197.145754] setup-gitlab-start[2227]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 197.526898] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:26 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 341 gitlab # [ 198.160792] gitaly[927]: time="2026-09-06T16:16:27.194Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.193" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.170692] gitaly[927]: time="2026-09-06T16:16:27.194Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.193" grpc.time_ms=0.546 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.285932] gitaly[927]: time="2026-09-06T16:16:27.319Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.128" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.319" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.286955] gitaly[927]: time="2026-09-06T16:16:27.320Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.128" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.319" grpc.time_ms=0.41 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.406768] gitaly[927]: time="2026-09-06T16:16:27.440Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 diskcache=9c61d3dd-5d21-4cd9-95f0-bd1661ff4a3d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:16:37.191" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.382" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.408679] gitaly[927]: time="2026-09-06T16:16:27.440Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1580 command.oublock=32 command.real_time_ms=27 command.spawn_token_fork_ms=1 command.system_time_ms=17 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:16:37.191" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.382" grpc.time_ms=58.183 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.459283] gitaly[927]: time="2026-09-06T16:16:27.492Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.301" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.492" pid=927 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 198.460983] gitaly[927]: time="2026-09-06T16:16:27.493Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.301" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.492" grpc.time_ms=0.658 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.492239] gitaly[927]: time="2026-09-06T16:16:27.525Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:16:37.013" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.513" grpc.time_ms=11.719 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.562578] gitaly[927]: time="2026-09-06T16:16:27.596Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.094" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.595" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.564543] gitaly[927]: time="2026-09-06T16:16:27.596Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.094" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.595" grpc.time_ms=0.577 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.651099] gitaly[927]: time="2026-09-06T16:16:27.684Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 diskcache=1456480e-c13a-4780-a07a-9a06e3394f7a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:16:37.144" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.644" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.670074] gitaly[927]: time="2026-09-06T16:16:27.685Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1570 command.oublock=32 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:16:37.144" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.644" grpc.time_ms=40.122 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.671341] gitaly[927]: time="2026-09-06T16:16:27.695Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.193" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.694" pid=927 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 198.672843] gitaly[927]: time="2026-09-06T16:16:27.695Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.193" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.694" grpc.time_ms=0.881 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.712161] gitaly[927]: time="2026-09-06T16:16:27.745Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=41 command.spawn_token_fork_ms=0 command.system_time_ms=23 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:16:37.201" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.702" grpc.time_ms=42.895 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.753540] gitaly[927]: time="2026-09-06T16:16:27.787Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:16:37.286" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.786" grpc.time_ms=0.155 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.763719] gitaly[927]: time="2026-09-06T16:16:27.797Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:16:37.006" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:27.796" grpc.time_ms=0.477 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.227807] gitaly[927]: time="2026-09-06T16:16:28.260Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3FET9QKRE8M9SQRCPA23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:16:38.059" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:28.249" grpc.time_ms=10.909 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 200.269811] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-09-06T16:16:29.285Z","message":"Starting cluster with 1 processes"} gitlab # [ 200.612335] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:29 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 3058 gitlab # [ 200.616832] setup-gitlab-start[2521]: {"id":1,"message":null,"error":null} gitlab # [ 206.027053] gitaly[927]: time="2026-09-06T16:16:35.060Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3QBGYNQ1QAF07ZFMQC5X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:16:45.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:35.049" grpc.time_ms=10.586 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.480673] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:35 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 840 gitlab # [ 206.485144] setup-gitlab-start[2227]: + mkdir -p /tmp/test-flake gitlab # [ 206.487610] setup-gitlab-start[2227]: + cd /tmp/test-flake gitlab # [ 206.488683] setup-gitlab-start[2227]: + git init -b master gitlab # [ 206.507319] setup-gitlab-start[2610]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 206.508382] setup-gitlab-start[2227]: + git config user.name test gitlab # [ 206.511433] setup-gitlab-start[2227]: + git config user.email test@example.com gitlab # [ 206.514556] setup-gitlab-start[2227]: + cat gitlab # [ 206.517082] setup-gitlab-start[2227]: + git add flake.nix gitlab # [ 206.520417] setup-gitlab-start[2227]: + git commit -m 'initial commit' gitlab # [ 206.543259] setup-gitlab-start[2615]: [master (root-commit) 1f01eb3] initial commit gitlab # [ 206.544329] setup-gitlab-start[2615]: 1 file changed, 10 insertions(+) gitlab # [ 206.546391] setup-gitlab-start[2615]: create mode 100644 flake.nix gitlab # [ 206.547383] setup-gitlab-start[2227]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 206.553968] setup-gitlab-start[2227]: + git push -u origin master gitlab # [ 206.629691] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 51 gitlab # [ 206.732581] gitaly[927]: time="2026-09-06T16:16:35.766Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RANJNKTVVE1B1ES1CGJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:35.754" grpc.time_ms=11.903 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 206.734785] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 96 gitlab # [ 207.007468] gitaly[927]: time="2026-09-06T16:16:36.041Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:16:46.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:36.030" grpc.time_ms=10.802 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.068358] gitaly[927]: time="2026-09-06T16:16:36.101Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:17:06.093" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:16:36.093" grpc.time_ms=7.801 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.090916] gitaly[927]: time="2026-09-06T16:16:36.124Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 duration_ms=147ns method=POST pid=927 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 207.113676] gitaly[927]: time="2026-09-06T16:16:36.147Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 duration_ms=21ns method=POST pid=927 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 207.121370] gitaly[927]: time="2026-09-06T16:16:36.147Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:16:35.975" grpc.time_ms=172.008 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 207.171546] gitaly[927]: time="2026-09-06T16:16:36.205Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:16:36.203" grpc.time_ms=1.685 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 207.217815] gitaly[927]: time="2026-09-06T16:16:36.251Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:16:36.251" grpc.time_ms=0.376 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 207.245597] gitaly[927]: time="2026-09-06T16:16:36.279Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:16:36.279" grpc.time_ms=0.258 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 207.370265] gitaly[927]: time="2026-09-06T16:16:36.403Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:16:46.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:36.391" grpc.time_ms=12.183 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.425642] gitaly[927]: time="2026-09-06T16:16:36.459Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1566 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:16:46.057" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:36.438" grpc.time_ms=20.889 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.454850] gitaly[927]: time="2026-09-06T16:16:36.488Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 duration_ms=180ns method=POST pid=927 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 207.458196] gitaly[927]: time="2026-09-06T16:16:36.492Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:16:36.307" grpc.time_ms=184.713 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 207.480633] gitaly[927]: time="2026-09-06T16:16:36.514Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 diskcache=38a33502-85b7-4c52-b70f-26c8d1964369 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:35.892" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 207.490223] gitaly[927]: time="2026-09-06T16:16:36.514Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=202 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13865 command.oublock=32 command.real_time_ms=611 command.spawn_token_fork_ms=0 command.system_time_ms=118 command.user_time_ms=84 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:35.892" grpc.time_ms=622.293 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 207.503154] gitlab-workhorse[824]: time="2026-09-06T16:16:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 207.503781] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:36 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 704 gitlab # [ 207.504664] setup-gitlab-start[2619]: To http://gitlab/root/test-flake.git gitlab # [ 207.505182] setup-gitlab-start[2619]: * [new branch] master -> master gitlab # [ 207.505668] setup-gitlab-start[2619]: branch 'master' set up to track 'origin/master'. gitlab # [ 207.510412] systemd[1]: Finished setup-gitlab.service. gitlab # [ 207.514625] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 208.93 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 208.823515] nixbot[2698]: {"ts": "2026-09-06T16:16:37.856Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 208.850419] nixbot[2698]: {"ts": "2026-09-06T16:16:37.883Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 208.885052] nixbot[2698]: {"ts": "2026-09-06T16:16:37.918Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 208.895786] nixbot[2698]: {"ts": "2026-09-06T16:16:37.929Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 208.906278] nixbot[2698]: {"ts": "2026-09-06T16:16:37.939Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 208.920716] nixbot[2698]: {"ts": "2026-09-06T16:16:37.954Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 208.930781] nixbot[2698]: {"ts": "2026-09-06T16:16:37.964Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 208.938716] nixbot[2698]: {"ts": "2026-09-06T16:16:37.972Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 208.951931] nixbot[2698]: {"ts": "2026-09-06T16:16:37.985Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 208.960085] nixbot[2698]: {"ts": "2026-09-06T16:16:37.993Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 208.974409] nixbot[2698]: {"ts": "2026-09-06T16:16:38.007Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 208.987429] nixbot[2698]: {"ts": "2026-09-06T16:16:38.020Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 208.997131] nixbot[2698]: {"ts": "2026-09-06T16:16:38.030Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 209.005772] nixbot[2698]: {"ts": "2026-09-06T16:16:38.039Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 209.016919] nixbot[2698]: {"ts": "2026-09-06T16:16:38.050Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 209.025476] nixbot[2698]: {"ts": "2026-09-06T16:16:38.059Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 209.033888] nixbot[2698]: {"ts": "2026-09-06T16:16:38.067Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 209.041648] nixbot[2698]: {"ts": "2026-09-06T16:16:38.075Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 209.053974] nixbot[2698]: {"ts": "2026-09-06T16:16:38.087Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 209.061935] nixbot[2698]: {"ts": "2026-09-06T16:16:38.095Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 209.070303] nixbot[2698]: {"ts": "2026-09-06T16:16:38.103Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 209.082492] nixbot[2698]: {"ts": "2026-09-06T16:16:38.116Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 209.090315] nixbot[2698]: {"ts": "2026-09-06T16:16:38.123Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 209.099583] nixbot[2698]: {"ts": "2026-09-06T16:16:38.133Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 209.106890] nixbot[2698]: {"ts": "2026-09-06T16:16:38.140Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 209.115435] nixbot[2698]: {"ts": "2026-09-06T16:16:38.149Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 209.123425] nixbot[2698]: {"ts": "2026-09-06T16:16:38.156Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 209.129723] nixbot[2698]: {"ts": "2026-09-06T16:16:38.163Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 209.136864] nixbot[2698]: {"ts": "2026-09-06T16:16:38.170Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 209.157185] nixbot[2698]: {"ts": "2026-09-06T16:16:38.190Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 209.168720] nixbot[2698]: {"ts": "2026-09-06T16:16:38.202Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 209.176596] nixbot[2698]: {"ts": "2026-09-06T16:16:38.210Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 209.532992] nixbot[2698]: INFO: Started server process [2698] gitlab # [ 209.535160] nixbot[2698]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 209.551955] nixbot[2698]: INFO: ::ffff:127.0.0.1:35456 - "GET /health HTTP/1.1" 200 OK gitlab # [ 209.556775] systemd[1]: Started nixbot CI service. gitlab # [ 209.558670] systemd[1]: Reached target Multi-User System. gitlab # [ 209.561274] systemd[1]: Startup finished in 1.305s (kernel) + 3.661s (initrd) + 3min 24.593s (userspace) = 3min 29.561s. gitlab # [ 209.736775] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 182 gitlab # [ 209.738501] nixbot[2698]: {"ts": "2026-09-06T16:16:38.770Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 209.762870] nixbot[2698]: {"ts": "2026-09-06T16:16:38.796Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 209.844158] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 71 gitlab # [ 209.847454] nixbot[2698]: {"ts": "2026-09-06T16:16:38.880Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 209.849051] nixbot[2698]: {"ts": "2026-09-06T16:16:38.882Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 209.944847] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 93 gitlab # [ 209.946541] nixbot[2698]: {"ts": "2026-09-06T16:16:38.980Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 210.063483] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 102 gitlab # [ 210.065176] nixbot[2698]: {"ts": "2026-09-06T16:16:39.098Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 210.200952] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 117 gitlab # [ 210.202663] nixbot[2698]: {"ts": "2026-09-06T16:16:39.236Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 210.257855] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 49 gitlab # [ 210.259679] nixbot[2698]: {"ts": "2026-09-06T16:16:39.293Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 210.319180] gitaly[927]: time="2026-09-06T16:16:39.352Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3VW5X520HVM3C3H3YY99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-06T16:17:09.339" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:39.340" grpc.time_ms=11.966 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.372194] gitaly[927]: time="2026-09-06T16:16:39.405Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3VW5X520HVM3C3H3YY99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:17:09.404" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:16:39.405" grpc.time_ms=0.443 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.386557] gitaly[927]: time="2026-09-06T16:16:39.420Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3VW5X520HVM3C3H3YY99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-06T16:16:49.223" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:16:39.414" grpc.time_ms=5.885 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: waiting for unit nixbot.service, in 2.11 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 210.437087] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 167 gitlab # [ 210.437692] nixbot[2698]: {"ts": "2026-09-06T16:16:39.469Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 210.441710] nixbot[2698]: INFO: ::ffff:127.0.0.1:35472 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 210.501065] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 60 gitlab # [ 210.505720] nixbot[2698]: {"ts": "2026-09-06T16:16:39.535Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 210.510988] nixbot[2698]: {"ts": "2026-09-06T16:16:39.538Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 210.546106] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 81 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 0.11 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 210.578860] gitaly[927]: time="2026-09-06T16:16:39.612Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3W4FQ51KZ0S5F99R02ZV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:39.603" grpc.time_ms=7.549 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.579616] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 44 gitlab # [ 210.668285] gitaly[927]: time="2026-09-06T16:16:39.702Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3W6BWXMMGMFQG8BA1RT8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:39.690" pid=927 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 210.669760] gitaly[927]: time="2026-09-06T16:16:39.703Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3W6BWXMMGMFQG8BA1RT8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:39.690" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.670966] gitaly[927]: time="2026-09-06T16:16:39.703Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3W6BWXMMGMFQG8BA1RT8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:39.690" grpc.time_ms=12.853 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.723819] gitlab-workhorse[824]: time="2026-09-06T16:16:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 210.724569] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 130 gitlab # [ 210.792304] gitaly[927]: time="2026-09-06T16:16:39.825Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WAMFYDW540H3AN7E6HD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:39.813" pid=927 remote_ip="2001:db8:1::1" request_sha=958626d6a90f896ba9a0484b59f5ce3f4889127f response_bytes=228 span.kind=server system=grpc gitlab # [ 210.794225] gitaly[927]: time="2026-09-06T16:16:39.826Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1206 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WAMFYDW540H3AN7E6HD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:39.813" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.824356] gitaly[927]: time="2026-09-06T16:16:39.857Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1206 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WAMFYDW540H3AN7E6HD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:39.813" grpc.time_ms=43.869 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.839259] gitlab-workhorse[824]: time="2026-09-06T16:16:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=228 gitlab # [ 210.840179] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 107 gitlab # [ 210.919304] gitaly[927]: time="2026-09-06T16:16:39.952Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WF19ZWDDNWSGPNNG3BX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:39.942" grpc.time_ms=9.964 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.921688] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 49 gitlab # [ 210.984407] gitaly[927]: time="2026-09-06T16:16:40.018Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WH2VYBZFF5WSPEZTER4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:40.007" pid=927 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 210.986274] gitaly[927]: time="2026-09-06T16:16:40.018Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WH2VYBZFF5WSPEZTER4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:40.007" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.014291] gitaly[927]: time="2026-09-06T16:16:40.047Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WH2VYBZFF5WSPEZTER4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:40.007" grpc.time_ms=39.496 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.015388] gitlab-workhorse[824]: time="2026-09-06T16:16:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 211.016120] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 77 gitlab # [ 211.110453] gitaly[927]: time="2026-09-06T16:16:40.144Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WN36M4BRWN0BNEYW1P3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:40.135" grpc.time_ms=8.185 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.135700] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 211.201938] gitaly[927]: time="2026-09-06T16:16:40.235Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WQCR1AWHV5RVT5DC7GV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:40.212" pid=927 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 211.203742] gitaly[927]: time="2026-09-06T16:16:40.235Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1167 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WQCR1AWHV5RVT5DC7GV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:40.212" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.206262] gitaly[927]: time="2026-09-06T16:16:40.236Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1167 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WQCR1AWHV5RVT5DC7GV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:40.212" grpc.time_ms=23.657 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.255688] gitlab-workhorse[824]: time="2026-09-06T16:16:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 211.256703] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 116 gitlab # [ 211.357844] gitaly[927]: time="2026-09-06T16:16:40.391Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:17:10.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:16:40.390" grpc.time_ms=0.607 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.431491] gitaly[927]: time="2026-09-06T16:16:40.465Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:17:10.156" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:40.456" grpc.time_ms=8.158 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.466102] gitaly[927]: time="2026-09-06T16:16:40.499Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:16:50.113" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:40.494" grpc.time_ms=5.54 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.498864] gitaly[927]: time="2026-09-06T16:16:40.532Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:16:50.146" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:40.527" grpc.time_ms=5.36 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.584604] gitaly[927]: time="2026-09-06T16:16:40.618Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:16:50.236" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:40.617" grpc.time_ms=0.33 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.600627] gitaly[927]: time="2026-09-06T16:16:40.634Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:16:50.245" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:40.626" grpc.time_ms=8.039 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.632135] gitaly[927]: time="2026-09-06T16:16:40.665Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:16:40.663" grpc.time_ms=0.449 pid=927 span.kind=server system=grpc gitlab # [ 211.661252] gitaly[927]: time="2026-09-06T16:16:40.694Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:16:40.694" grpc.time_ms=0.692 pid=927 span.kind=server system=grpc gitlab # [ 211.679236] gitaly[927]: time="2026-09-06T16:16:40.713Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z diskcache=449b2698-bb84-4c99-b264-362adf1e3f9b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:16:50.256" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:40.636" pid=927 ref=refs/keep-around/1f01eb39b91f0d35d859c09f4fef763afded0209 remote_ip="2001:db8:1::1" revision=1f01eb39b91f0d35d859c09f4fef763afded0209 span.kind=server system=grpc username=root gitlab # [ 211.679729] gitaly[927]: time="2026-09-06T16:16:40.713Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=49 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5405 command.oublock=8 command.real_time_ms=67 command.spawn_token_fork_ms=0 command.system_time_ms=33 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:16:50.256" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:16:40.636" grpc.time_ms=76.575 pid=927 ref=refs/keep-around/1f01eb39b91f0d35d859c09f4fef763afded0209 remote_ip="2001:db8:1::1" revision=1f01eb39b91f0d35d859c09f4fef763afded0209 span.kind=server system=grpc username=root gitlab # [ 211.682874] gitaly[927]: time="2026-09-06T16:16:40.716Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:17:10.415" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:16:40.716" grpc.time_ms=0.303 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.789926] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 487 gitlab # [ 211.791793] nixbot[2698]: {"ts": "2026-09-06T16:16:40.825Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 \"HTTP/1.1 201 Created\""} gitlab # [ 211.802669] nixbot[2698]: {"ts": "2026-09-06T16:16:40.836Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-1f01eb39b91f-0"} gitlab # [ 212.003098] nixbot[2698]: {"ts": "2026-09-06T16:16:41.036Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-1f01eb39b91f-0", "argv0": "bwrap"} gitlab # [ 212.100314] systemd[1]: Started Nix Daemon. gitlab # [ 212.174684] nix-daemon[2858]: accepted connection from pid 2853, user nixbot gitlab # [ 217.169088] nix-daemon[2858]: accepted connection from pid 2907, user nixbot gitlab # [ 217.186827] nix-daemon[2858]: accepted connection from pid 2853, user nixbot gitlab # [ 221.405203] nix-daemon[2858]: accepted connection from pid 2853, user nixbot gitlab # [ 221.496103] nix-daemon[2858]: accepted connection from pid 2949, user nixbot gitlab # [ 221.548162] gitaly[927]: time="2026-09-06T16:16:50.581Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR46TQ4N97XAKMFHG9242Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:17:20.579" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:16:50.580" grpc.time_ms=0.72 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.637526] gitaly[927]: time="2026-09-06T16:16:50.671Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR46TQ4N97XAKMFHG9242Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:00.161" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:50.661" grpc.time_ms=9.105 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.715722] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 228 gitlab # [ 221.724625] nixbot[2698]: {"ts": "2026-09-06T16:16:50.757Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 \"HTTP/1.1 201 Created\""} gitlab # [ 221.782933] gitaly[927]: time="2026-09-06T16:16:50.816Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR472CXMD38TGCP3B837JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:17:20.815" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:16:50.815" grpc.time_ms=0.838 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.824173] gitaly[927]: time="2026-09-06T16:16:50.857Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR472CXMD38TGCP3B837JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:00.059" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:50.849" grpc.time_ms=7.86 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.862786] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 132 gitlab # [ 221.868586] nixbot[2698]: {"ts": "2026-09-06T16:16:50.902Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 \"HTTP/1.1 201 Created\""} gitlab # [ 226.614564] nix-daemon[2858]: accepted connection from pid 3024, user nixbot gitlab # [ 226.709587] gitaly[927]: time="2026-09-06T16:16:55.742Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR4BW2BJTQ1SWD3NKP7RV2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:17:25.740" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:16:55.741" grpc.time_ms=0.862 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 226.754087] gitaly[927]: time="2026-09-06T16:16:55.787Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR4BW2BJTQ1SWD3NKP7RV2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:05.278" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:16:55.779" grpc.time_ms=8.611 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 226.824834] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 175 gitlab # [ 226.831835] nixbot[2698]: {"ts": "2026-09-06T16:16:55.864Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 \"HTTP/1.1 201 Created\""} gitlab # [ 227.165663] sidekiq-cluster[2565]: 2026-09-06T16:16:56.199Z pid=2565 tid=xbh INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2565, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>436789248} gitlab # [ 227.178678] gitaly[927]: time="2026-09-06T16:16:56.212Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M1VR4CAZ299YZFX5AQV5SMMZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=200 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:56.195" grpc.time_ms=15.997 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 227.183774] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 57 gitlab # [ 227.237176] gitaly[927]: time="2026-09-06T16:16:56.271Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR4CCY06F7XZ2GJZ66GJND grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:56.265" pid=927 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 227.246603] gitaly[927]: time="2026-09-06T16:16:56.280Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR4CCY06F7XZ2GJZ66GJND grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:56.265" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 227.258207] gitaly[927]: time="2026-09-06T16:16:56.280Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR4CCY06F7XZ2GJZ66GJND grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:56.265" grpc.time_ms=14.907 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 227.270683] gitlab-workhorse[824]: time="2026-09-06T16:16:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 227.271398] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 58 gitlab # [ 227.328305] gitaly[927]: time="2026-09-06T16:16:56.362Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR4CG09PEDSS4B39V6MDH2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:56.356" grpc.time_ms=5.981 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 227.340230] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 53 gitlab # [ 227.380304] gitaly[927]: time="2026-09-06T16:16:56.414Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR4CHS9Z2Z18XWTYSZRYE4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:56.408" pid=927 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 227.380846] gitaly[927]: time="2026-09-06T16:16:56.414Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR4CHS9Z2Z18XWTYSZRYE4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:56.408" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 227.394308] sidekiq-cluster[2565]: W, [2026-09-06T16:16:56.428095 #2565] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 227.402265] gitaly[927]: time="2026-09-06T16:16:56.436Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR4CHS9Z2Z18XWTYSZRYE4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:16:56.408" grpc.time_ms=27.763 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 227.416169] gitlab-workhorse[824]: time="2026-09-06T16:16:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 227.418298] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:16:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 72 gitlab # [ 227.437637] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:16:56.471Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 229.465124] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:16:58.497Z","message":"GitLab reliable fetch activated!"} gitlab # [ 229.469135] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:16:58.498Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 252.161557] postgres[3272]: [3272] ERROR: relation "table_without_models" does not exist gitlab # [ 252.164604] postgres[3272]: [3272] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 254.745420] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:23.778Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 254.747160] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:23.780Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 254.747706] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:23.780Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 254.748805] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:23.780Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 254.749956] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:23.783Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 44.65 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 255.284816] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:24 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 29 gitlab # [ 255.370947] gitaly[927]: time="2026-09-06T16:17:24.403Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57TYJGW28Y8SCKBQA1P0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:24.394" grpc.time_ms=8.879 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 255.372840] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:24 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 85 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 255.592458] gitaly[927]: time="2026-09-06T16:17:24.625Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:17:54.317" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:17:24.618" grpc.time_ms=7.739 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.640490] gitaly[927]: time="2026-09-06T16:17:24.674Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH duration_ms=102ns method=POST pid=927 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 255.662248] gitaly[927]: time="2026-09-06T16:17:24.696Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH duration_ms=21ns method=POST pid=927 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 255.669708] gitaly[927]: time="2026-09-06T16:17:24.696Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:24.571" grpc.time_ms=125.047 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 255.717724] gitaly[927]: time="2026-09-06T16:17:24.751Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:24.751" grpc.time_ms=0.433 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 255.757180] gitaly[927]: time="2026-09-06T16:17:24.790Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:24.790" grpc.time_ms=0.547 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 255.782179] gitaly[927]: time="2026-09-06T16:17:24.815Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:24.815" grpc.time_ms=0.287 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 255.864883] gitaly[927]: time="2026-09-06T16:17:24.898Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH duration_ms=49ns method=POST pid=927 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 255.872829] gitaly[927]: time="2026-09-06T16:17:24.904Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:24.849" grpc.time_ms=55.236 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 255.894083] gitaly[927]: time="2026-09-06T16:17:24.927Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH diskcache=1854510f-8f91-4da7-97d2-c75eb1be9197 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:24.518" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 255.895210] gitaly[927]: time="2026-09-06T16:17:24.927Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=170 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13168 command.oublock=32 command.real_time_ms=402 command.spawn_token_fork_ms=0 command.system_time_ms=128 command.user_time_ms=42 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:24.518" grpc.time_ms=409.007 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 255.896111] gitlab-workhorse[824]: time="2026-09-06T16:17:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 255.896838] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:24 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 468 gitlab # To http://gitlab/root/test-flake.git gitlab # 1f01eb3..15ca3fb master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.73 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d' gitlab # [ 256.003543] nixbot[2698]: INFO: ::ffff:127.0.0.1:59502 - "GET /api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d', in 0.08 seconds) gitlab # [ 256.489502] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:25.522Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b7bf422c8fd509ecf12ba336","created_at":"2026-09-06T16:16:27.540Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:27.577Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b7bf422c8fd509ecf12ba336: start","job_status":"start","queue_duration_s":57.944728,"scheduling_latency_s":57.944778} gitlab # [ 256.687460] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:25.720Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b7bf422c8fd509ecf12ba336","created_at":"2026-09-06T16:16:27.540Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:27.577Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b7bf422c8fd509ecf12ba336: done: 0.198736 sec","job_status":"done","queue_duration_s":57.944728,"scheduling_latency_s":57.944778,"redis_calls":24,"redis_duration_s":0.00603,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.003303,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09453911,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.198736,"completed_at":"2026-09-06T16:17:25.720Z","load_balancing_strategy":"primary","db_duration_s":0.064136,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.690465] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:25.722Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"4bf8a9770ba13f51ca4e4668","created_at":"2026-09-06T16:16:27.586Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:27.591Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-4bf8a9770ba13f51ca4e4668: start","job_status":"start","queue_duration_s":58.131235,"scheduling_latency_s":58.131256} gitlab # [ 256.777474] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:25.810Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"4bf8a9770ba13f51ca4e4668","created_at":"2026-09-06T16:16:27.586Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:27.591Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-4bf8a9770ba13f51ca4e4668: done: 0.088398 sec","job_status":"done","queue_duration_s":58.131235,"scheduling_latency_s":58.131256,"redis_calls":14,"redis_duration_s":0.003396,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000669,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001596,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001131,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.069173596,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.088398,"completed_at":"2026-09-06T16:17:25.810Z","load_balancing_strategy":"primary","db_duration_s":0.024604,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.779967] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:25.812Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"bc56fd117212e6ed2f40a216","created_at":"2026-09-06T16:16:28.138Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:28.144Z","job_size_bytes":5,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-bc56fd117212e6ed2f40a216: start","job_status":"start","queue_duration_s":57.66804,"scheduling_latency_s":57.668059} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d' gitlab # [ 256.950525] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:25.983Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"bc56fd117212e6ed2f40a216","created_at":"2026-09-06T16:16:28.138Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:28.144Z","job_size_bytes":5,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-bc56fd117212e6ed2f40a216: done: 0.171272 sec","job_status":"done","queue_duration_s":57.66804,"scheduling_latency_s":57.668059,"redis_calls":12,"redis_duration_s":0.0029460000000000003,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000767,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001744,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.133062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000359543,"duration_s":0.171272,"completed_at":"2026-09-06T16:17:25.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029346,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d', in 0.03 seconds) gitlab # [ 257.028856] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:26.062Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"3076d7981f8a8ba9b137e05d","created_at":"2026-09-06T16:16:28.167Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:28.171Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-3076d7981f8a8ba9b137e05d: start","job_status":"start","queue_duration_s":57.890535,"scheduling_latency_s":57.890583} gitlab # [ 257.034562] nixbot[2698]: INFO: ::ffff:127.0.0.1:59512 - "GET /api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 200 OK gitlab # [ 257.235842] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:26.269Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"3076d7981f8a8ba9b137e05d","created_at":"2026-09-06T16:16:28.167Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:28.171Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-3076d7981f8a8ba9b137e05d: done: 0.207356 sec","job_status":"done","queue_duration_s":57.890535,"scheduling_latency_s":57.890583,"redis_calls":11,"redis_duration_s":0.002981,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000812,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001638,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.172,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.172,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000495035,"duration_s":0.207356,"completed_at":"2026-09-06T16:17:26.269Z","load_balancing_strategy":"primary","db_duration_s":0.081501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.315903] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:26.349Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"142ae231429692deb4284918","created_at":"2026-09-06T16:16:28.174Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:28.177Z","job_size_bytes":89,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-142ae231429692deb4284918: start","job_status":"start","queue_duration_s":58.171419,"scheduling_latency_s":58.171455} gitlab # [ 257.370811] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:26.403Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"142ae231429692deb4284918","created_at":"2026-09-06T16:16:28.174Z","trace_propagation_headers":{"sentry-trace":"e5c6cbca9f4d47b196feb457ad5fba33-6b1c8c9d3ed8440b","baggage":"sentry-trace_id=e5c6cbca9f4d47b196feb457ad5fba33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3FET9QKRE8M9SQRCPA23","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:28.177Z","job_size_bytes":89,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-142ae231429692deb4284918: done: 0.054743 sec","job_status":"done","queue_duration_s":58.171419,"scheduling_latency_s":58.171455,"redis_calls":11,"redis_duration_s":0.003108,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001082,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001554,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000445029,"duration_s":0.054743,"completed_at":"2026-09-06T16:17:26.403Z","load_balancing_strategy":"primary","db_duration_s":0.007383,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.426077] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:26.459Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 1f01eb39b91f0d35d859c09f4fef763afded0209 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"536237f5f910ea948393d5a2","created_at":"2026-09-06T16:16:36.349Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-3de122cfdd4c4de1","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:36.366Z","job_size_bytes":185,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-536237f5f910ea948393d5a2: start","job_status":"start","queue_duration_s":50.092353,"scheduling_latency_s":50.092389} gitlab # [ 257.442453] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:26.476Z","message":"changes: \"0000000000000000000000000000000000000000 1f01eb39b91f0d35d859c09f4fef763afded0209 refs/heads/master\\n\""} gitlab # [ 257.476126] gitaly[927]: time="2026-09-06T16:17:26.509Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:36.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:26.503" grpc.time_ms=6.189 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.498572] gitaly[927]: time="2026-09-06T16:17:26.532Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:17:36.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:26.522" grpc.time_ms=10.071 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.562465] gitaly[927]: time="2026-09-06T16:17:26.594Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:17:36.085" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:26.586" grpc.time_ms=8.7 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.609412] gitaly[927]: time="2026-09-06T16:17:26.643Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:26.642" grpc.time_ms=0.544 pid=927 span.kind=server system=grpc gitlab # [ 257.652663] gitaly[927]: time="2026-09-06T16:17:26.686Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:26.685" grpc.time_ms=0.607 pid=927 span.kind=server system=grpc gitlab # [ 257.675721] gitaly[927]: time="2026-09-06T16:17:26.709Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 diskcache=029ba16e-117e-4f00-9fa7-f52145f987ff grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:17:36.104" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:26.605" pid=927 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 257.685245] gitaly[927]: time="2026-09-06T16:17:26.709Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=93 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5124 command.oublock=8 command.real_time_ms=91 command.spawn_token_fork_ms=0 command.system_time_ms=53 command.user_time_ms=40 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:17:36.104" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:26.605" grpc.time_ms=104.52 pid=927 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 257.694929] gitaly[927]: time="2026-09-06T16:17:26.728Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:17:36.228" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:26.728" grpc.time_ms=0.143 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.712406] gitaly[927]: time="2026-09-06T16:17:26.746Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=936 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:17:36.232" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:26.732" grpc.time_ms=13.385 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.785616] gitaly[927]: time="2026-09-06T16:17:26.819Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=340 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-06T16:17:56.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:26.811" grpc.time_ms=7.555 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.822918] gitaly[927]: time="2026-09-06T16:17:26.856Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:17:36.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:26.856" grpc.time_ms=0.305 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.850135] gitaly[927]: time="2026-09-06T16:17:26.883Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:17:36.026" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:17:26.876" grpc.time_ms=7.253 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.877207] gitaly[927]: time="2026-09-06T16:17:26.910Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:36.050" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:26.900" grpc.time_ms=10.209 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.897345] gitaly[927]: time="2026-09-06T16:17:26.931Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:36.072" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:26.923" grpc.time_ms=7.947 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.928700] gitaly[927]: time="2026-09-06T16:17:26.962Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:17:56.146" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:26.947" grpc.time_ms=15.103 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d' gitlab # [ 258.076133] nixbot[2698]: INFO: ::ffff:127.0.0.1:59528 - "GET /api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d', in 0.05 seconds) gitlab # [ 258.293147] gitaly[927]: time="2026-09-06T16:17:27.326Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:17:37.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:27.325" grpc.time_ms=1.242 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.365301] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:27.397Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 1f01eb39b91f0d35d859c09f4fef763afded0209 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"536237f5f910ea948393d5a2","created_at":"2026-09-06T16:16:36.349Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-3de122cfdd4c4de1","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:16:36.366Z","job_size_bytes":185,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-536237f5f910ea948393d5a2: done: 0.938415 sec","job_status":"done","queue_duration_s":50.092353,"scheduling_latency_s":50.092389,"gitaly_calls":13,"gitaly_duration_s":0.277465,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.07136300000000001,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.00411,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006309,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001515,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004749,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.018828,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.035852,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.19,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.212,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.516035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000593651,"exclusive_lock_hold_duration_s":0.026972981,"duration_s":0.938415,"completed_at":"2026-09-06T16:17:27.397Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.099771,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 258.443575] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:27.476Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e9682aafe2098424fea07049","created_at":"2026-09-06T16:16:40.724Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-b2a9f12b90f74447","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:40.729Z","job_size_bytes":65,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e9682aafe2098424fea07049: start","job_status":"start","queue_duration_s":46.747513,"scheduling_latency_s":46.747568} gitlab # [ 258.538975] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:27.572Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e9682aafe2098424fea07049","created_at":"2026-09-06T16:16:40.724Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-b2a9f12b90f74447","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:40.729Z","job_size_bytes":65,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e9682aafe2098424fea07049: done: 0.095743 sec","job_status":"done","queue_duration_s":46.747513,"scheduling_latency_s":46.747568,"redis_calls":16,"redis_duration_s":0.00432,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002287,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000301993,"duration_s":0.095743,"completed_at":"2026-09-06T16:17:27.572Z","load_balancing_strategy":"primary","db_duration_s":0.024407,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 258.614589] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:27.647Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1663a4501d1e206d9f376993","created_at":"2026-09-06T16:16:40.729Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-b2a9f12b90f74447","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:40.732Z","job_size_bytes":65,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1663a4501d1e206d9f376993: start","job_status":"start","queue_duration_s":46.915075,"scheduling_latency_s":46.915128} gitlab # [ 258.731399] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:27.764Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1663a4501d1e206d9f376993","created_at":"2026-09-06T16:16:40.729Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-b2a9f12b90f74447","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:40.732Z","job_size_bytes":65,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1663a4501d1e206d9f376993: done: 0.11705 sec","job_status":"done","queue_duration_s":46.915075,"scheduling_latency_s":46.915128,"redis_calls":21,"redis_duration_s":0.006073,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002683,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001936,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001454,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000546439,"duration_s":0.11705,"completed_at":"2026-09-06T16:17:27.764Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.802207] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:27.835Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"635f14b805cf1c0a64fe1ca8","created_at":"2026-09-06T16:16:40.806Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-b2a9f12b90f74447","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:40.810Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-635f14b805cf1c0a64fe1ca8: start","job_status":"start","queue_duration_s":47.024555,"scheduling_latency_s":47.024608} gitlab # [ 259.005720] gitaly[927]: time="2026-09-06T16:17:28.038Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:28.037" grpc.time_ms=1.504 pid=927 span.kind=server system=grpc gitlab # [ 259.066424] gitaly[927]: time="2026-09-06T16:17:28.100Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:28.098" grpc.time_ms=1.462 pid=927 span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d', in 0.03 seconds) gitlab # [ 259.109333] gitaly[927]: time="2026-09-06T16:17:28.143Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:28.142" grpc.time_ms=0.528 pid=927 span.kind=server system=grpc gitlab # [ 259.113297] nixbot[2698]: INFO: ::ffff:127.0.0.1:59534 - "GET /api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 200 OK gitlab # [ 259.137285] gitaly[927]: time="2026-09-06T16:17:28.171Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z diskcache=b661a44f-8465-434e-be10-011c9f7ad090 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:17:57.194" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:27.995" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.146366] gitaly[927]: time="2026-09-06T16:17:28.171Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=164 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7479 command.oublock=0 command.real_time_ms=159 command.spawn_token_fork_ms=0 command.system_time_ms=108 command.user_time_ms=56 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:17:57.194" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:27.995" grpc.time_ms=175.773 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.332807] gitaly[927]: time="2026-09-06T16:17:28.366Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:17:58.064" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:28.365" grpc.time_ms=1.217 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.380366] gitaly[927]: time="2026-09-06T16:17:28.414Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3WWGGZG02XKX3MSRR92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:17:58.112" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:28.412" grpc.time_ms=0.997 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.410846] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.443Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"635f14b805cf1c0a64fe1ca8","created_at":"2026-09-06T16:16:40.806Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-b2a9f12b90f74447","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:40.810Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2565,"job_status":"rescheduled","message":"PipelineProcessWorker JID-635f14b805cf1c0a64fe1ca8: rescheduled"} gitlab # [ 259.437145] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.451Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"635f14b805cf1c0a64fe1ca8","created_at":"2026-09-06T16:16:40.806Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-b2a9f12b90f74447","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:16:40.810Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-635f14b805cf1c0a64fe1ca8: done: 0.61634 sec","job_status":"done","queue_duration_s":47.024555,"scheduling_latency_s":47.024608,"gitaly_calls":3,"gitaly_duration_s":0.186729,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.11597800000000001,"redis_read_bytes":460,"redis_write_bytes":29292,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001855,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002088,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009889,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004279,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15579,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.09216,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000543,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000692,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002617,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.08,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.023,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.029,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.05,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.310206,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.523306175,"duration_s":0.61634,"completed_at":"2026-09-06T16:17:28.451Z","load_balancing_strategy":"primary","db_duration_s":0.095104,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 259.478620] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.452Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","1f01eb39b91f0d35d859c09f4fef763afded0209 15ca3fb256fc690a0d60f7cdef964df568aa183d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7b35664f96dce83a93a45513","created_at":"2026-09-06T16:17:24.875Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-ec49468a0afa4100","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:17:24.877Z","job_size_bytes":185,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7b35664f96dce83a93a45513: start","job_status":"start","queue_duration_s":3.575129,"scheduling_latency_s":3.575148} gitlab # [ 259.479672] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.462Z","message":"changes: \"1f01eb39b91f0d35d859c09f4fef763afded0209 15ca3fb256fc690a0d60f7cdef964df568aa183d refs/heads/master\\n\""} gitlab # [ 259.511278] gitaly[927]: time="2026-09-06T16:17:28.545Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:38.215" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:28.535" grpc.time_ms=9.512 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.527066] gitaly[927]: time="2026-09-06T16:17:28.560Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:17:38.050" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:28.551" grpc.time_ms=9.757 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.539401] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.564Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e4e7dd1c2def7e814991c461","created_at":"2026-09-06T16:17:28.562Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-6319f31077c845e3","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"856a80a07a06d32b80242d0a","job_size_bytes":46,"pid":2565,"job_status":"deduplicated","message":"ProjectCacheWorker JID-e4e7dd1c2def7e814991c461: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 259.554754] gitaly[927]: time="2026-09-06T16:17:28.588Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:17:38.076" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:28.577" grpc.time_ms=11.432 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.570817] gitaly[927]: time="2026-09-06T16:17:28.604Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-06T16:17:58.291" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:28.592" grpc.time_ms=12.294 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.585365] gitaly[927]: time="2026-09-06T16:17:28.617Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:17:38.113" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:17:28.613" grpc.time_ms=4.268 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.608237] gitaly[927]: time="2026-09-06T16:17:28.641Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:38.131" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:28.632" grpc.time_ms=9.71 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.679542] gitaly[927]: time="2026-09-06T16:17:28.713Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:17:38.211" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:28.712" grpc.time_ms=0.759 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.704513] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.737Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","1f01eb39b91f0d35d859c09f4fef763afded0209 15ca3fb256fc690a0d60f7cdef964df568aa183d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7b35664f96dce83a93a45513","created_at":"2026-09-06T16:17:24.875Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-ec49468a0afa4100","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:17:24.877Z","job_size_bytes":185,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7b35664f96dce83a93a45513: done: 0.284919 sec","job_status":"done","queue_duration_s":3.575129,"scheduling_latency_s":3.575148,"gitaly_calls":7,"gitaly_duration_s":0.140036,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.02058,"redis_read_bytes":501,"redis_write_bytes":11940,"redis_cache_calls":9,"redis_cache_duration_s":0.002099,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.00086,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5717,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002449,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006461,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008711,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.107963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005456,"exclusive_lock_hold_duration_s":0.076994143,"duration_s":0.284919,"completed_at":"2026-09-06T16:17:28.737Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027251,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 259.709521] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.739Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"856a80a07a06d32b80242d0a","created_at":"2026-09-06T16:17:26.547Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:26.564Z","job_size_bytes":46,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-856a80a07a06d32b80242d0a: start","job_status":"start","queue_duration_s":2.175234,"scheduling_latency_s":2.17526} gitlab # [ 259.751697] gitaly[927]: time="2026-09-06T16:17:28.785Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-06T22:17:28.782" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:28.782" grpc.time_ms=2.319 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.774079] gitaly[927]: time="2026-09-06T16:17:28.807Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:17:58.497" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:28.797" grpc.time_ms=9.968 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.802075] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.834Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"856a80a07a06d32b80242d0a","created_at":"2026-09-06T16:17:26.547Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:26.564Z","job_size_bytes":46,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-856a80a07a06d32b80242d0a: done: 0.095133 sec","job_status":"done","queue_duration_s":2.175234,"scheduling_latency_s":2.17526,"gitaly_calls":2,"gitaly_duration_s":0.017484,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007159,"redis_read_bytes":69,"redis_write_bytes":4883,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001389,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000501,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001575,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003001,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000693,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000270146,"duration_s":0.095133,"completed_at":"2026-09-06T16:17:28.834Z","load_balancing_strategy":"primary","db_duration_s":0.016565,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 259.843910] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.836Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ba837407530a9d8a233800b4","created_at":"2026-09-06T16:17:26.578Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:26.582Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ba837407530a9d8a233800b4: start","job_status":"start","queue_duration_s":2.254771,"scheduling_latency_s":2.254833} gitlab # [ 259.896510] gitaly[927]: time="2026-09-06T16:17:28.930Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 diskcache=c487864a-ec68-494a-8554-cc8647380fde grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:17:28.895" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:28.895" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.897868] gitaly[927]: time="2026-09-06T16:17:28.930Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1280 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:17:28.895" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:28.895" grpc.time_ms=34.621 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.953937] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.986Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ba837407530a9d8a233800b4","created_at":"2026-09-06T16:17:26.578Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:26.582Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ba837407530a9d8a233800b4: done: 0.149202 sec","job_status":"done","queue_duration_s":2.254771,"scheduling_latency_s":2.254833,"gitaly_calls":1,"gitaly_duration_s":0.07687,"redis_calls":15,"redis_duration_s":0.003945,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000461,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001457,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.029,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.125141044,"duration_s":0.149202,"completed_at":"2026-09-06T16:17:28.986Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.956816] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:28.989Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"b12d19e0e66b4d3d2393c402","created_at":"2026-09-06T16:17:26.722Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:26.725Z","job_size_bytes":23,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b12d19e0e66b4d3d2393c402: start","job_status":"start","queue_duration_s":2.263867,"scheduling_latency_s":2.263895} gitlab # [ 260.008702] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:29.037Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"b12d19e0e66b4d3d2393c402","created_at":"2026-09-06T16:17:26.722Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:26.725Z","job_size_bytes":23,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b12d19e0e66b4d3d2393c402: done: 0.048515 sec","job_status":"done","queue_duration_s":2.263867,"scheduling_latency_s":2.263895,"gitaly_calls":1,"gitaly_duration_s":0.012687,"redis_calls":16,"redis_duration_s":0.004165,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000311,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001297,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001719,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048515,"completed_at":"2026-09-06T16:17:29.037Z","load_balancing_strategy":"primary","db_duration_s":0.003669,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 260.014959] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:29.039Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"8e7f12ad7dc6421d724da3b7","created_at":"2026-09-06T16:17:27.201Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:abb501110e10f8c5a42c850348f4a75860077e7cb3f47251ad5c28bf6f5354c7","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:27.205Z","job_size_bytes":467,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-8e7f12ad7dc6421d724da3b7: start","job_status":"start","queue_duration_s":1.833567,"scheduling_latency_s":1.833587} gitlab # [ 260.068860] gitaly[927]: time="2026-09-06T16:17:29.029Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:17:59.018" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:29.019" grpc.time_ms=9.953 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.069894] gitaly[927]: time="2026-09-06T16:17:29.061Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:17:59.060" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:29.060" grpc.time_ms=0.701 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.070893] gitaly[927]: time="2026-09-06T16:17:29.072Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:17:39.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:29.063" grpc.time_ms=8.76 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d' gitlab # [ 260.131454] gitaly[927]: time="2026-09-06T16:17:29.165Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:17:59.163" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:29.163" grpc.time_ms=1.029 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.145270] nixbot[2698]: INFO: ::ffff:127.0.0.1:59538 - "GET /api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d', in 0.03 seconds) gitlab # [ 260.153829] gitaly[927]: time="2026-09-06T16:17:29.187Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=189 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:17:59.183" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:29.183" grpc.time_ms=3.527 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.236029] gitaly[927]: time="2026-09-06T16:17:29.269Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:39.199" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:29.259" grpc.time_ms=10.046 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.731492] gitaly[927]: time="2026-09-06T16:17:29.765Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:17:59.751" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:29.752" grpc.time_ms=12.851 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.779536] gitaly[927]: time="2026-09-06T16:17:29.813Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR3RF27CH3HMPZEPBB39M5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:17:59.801" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:29.801" grpc.time_ms=11.345 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.829672] sidekiq-cluster[2565]: {"severity":"WARN","time":"2026-09-06T16:17:29.862Z","class":"CreatePipelineWorker","correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"1f01eb39b91f0d35d859c09f4fef763afded0209","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"1f01eb39b91f0d35d859c09f4fef763afded0209","ref":"refs/heads/master"},"retry":0} gitlab # [ 260.850380] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:29.883Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"8e7f12ad7dc6421d724da3b7","created_at":"2026-09-06T16:17:27.201Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:abb501110e10f8c5a42c850348f4a75860077e7cb3f47251ad5c28bf6f5354c7","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:27.205Z","job_size_bytes":467,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-8e7f12ad7dc6421d724da3b7: done: 0.844736 sec","job_status":"done","queue_duration_s":1.833567,"scheduling_latency_s":1.833587,"gitaly_calls":7,"gitaly_duration_s":0.14443,"redis_calls":51,"redis_duration_s":0.026568,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001098,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005276,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002906,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.628264,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000305905,"duration_s":0.844736,"completed_at":"2026-09-06T16:17:29.883Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048822,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 260.895250] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:29.928Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"7b27945c-6076-4ae9-b9b6-726711be1b2f\"}"],"class":"WebHookWorker","jid":"fb719291c989122e08e10eba","created_at":"2026-09-06T16:17:27.299Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:48386e67b75a897de2019a9f58cc428ae0d99459c297dd797d9d11e72a392f49","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:27.304Z","job_size_bytes":1751,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-fb719291c989122e08e10eba: start","job_status":"start","queue_duration_s":2.623993,"scheduling_latency_s":2.624021} gitlab # [ 261.038561] nixbot[2698]: INFO: ::ffff:127.0.0.1:59546 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 261.067855] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.100Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"7b27945c-6076-4ae9-b9b6-726711be1b2f\"}"],"class":"WebHookWorker","jid":"fb719291c989122e08e10eba","created_at":"2026-09-06T16:17:27.299Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-584533906a0f40a8","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:48386e67b75a897de2019a9f58cc428ae0d99459c297dd797d9d11e72a392f49","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:27.304Z","job_size_bytes":1751,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-fb719291c989122e08e10eba: done: 0.17159 sec","job_status":"done","queue_duration_s":2.623993,"scheduling_latency_s":2.624021,"redis_calls":19,"redis_duration_s":0.004246,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000914,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000764,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001825,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06540747500002908,"cpu_s":0.075887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000572698,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.17159,"completed_at":"2026-09-06T16:17:30.100Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.132564] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.165Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8a542a628224957a7b53734a","created_at":"2026-09-06T16:17:27.981Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:27.985Z","job_size_bytes":23,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8a542a628224957a7b53734a: start","job_status":"start","queue_duration_s":2.179872,"scheduling_latency_s":2.179908} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d' gitlab # [ 261.167423] gitaly[927]: time="2026-09-06T16:17:30.200Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5DFH9FEMC3DBT2GPDCVK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:30.192" grpc.time_ms=7.737 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 261.172631] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:30 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 105 gitlab # [ 261.196855] nixbot[2698]: INFO: ::ffff:127.0.0.1:59552 - "GET /api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d', in 0.05 seconds) gitlab # [ 261.225297] gitaly[927]: time="2026-09-06T16:17:30.258Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5DJXV273G9X3Q88P6VY8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:30.247" pid=927 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 261.226496] gitaly[927]: time="2026-09-06T16:17:30.259Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5DJXV273G9X3Q88P6VY8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:30.247" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 261.227436] gitaly[927]: time="2026-09-06T16:17:30.259Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5DJXV273G9X3Q88P6VY8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:30.247" grpc.time_ms=11.852 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 261.234749] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8a542a628224957a7b53734a","created_at":"2026-09-06T16:17:27.981Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:27.985Z","job_size_bytes":23,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8a542a628224957a7b53734a: done: 0.101989 sec","job_status":"done","queue_duration_s":2.179872,"scheduling_latency_s":2.179908,"redis_calls":10,"redis_duration_s":0.004334,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00079,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002989,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000555,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000310654,"duration_s":0.101989,"completed_at":"2026-09-06T16:17:30.267Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022181,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 261.290190] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.274Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b65d6487bc6c00dcbd1336a5","created_at":"2026-09-06T16:17:28.179Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.186Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-b65d6487bc6c00dcbd1336a5: start","job_status":"start","queue_duration_s":2.087949,"scheduling_latency_s":2.087967} gitlab # [ 261.290852] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.302Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b65d6487bc6c00dcbd1336a5","created_at":"2026-09-06T16:17:28.179Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.186Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-b65d6487bc6c00dcbd1336a5: done: 0.02769 sec","job_status":"done","queue_duration_s":2.087949,"scheduling_latency_s":2.087967,"redis_calls":10,"redis_duration_s":0.004914,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000749,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002974,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001191,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000572978,"duration_s":0.02769,"completed_at":"2026-09-06T16:17:30.302Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003726,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.342983] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.303Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2d2fe9b7c4c96ab66de4820a","created_at":"2026-09-06T16:17:28.202Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.208Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2d2fe9b7c4c96ab66de4820a: start","job_status":"start","queue_duration_s":2.094725,"scheduling_latency_s":2.094739} gitlab # [ 261.344178] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.325Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2d2fe9b7c4c96ab66de4820a","created_at":"2026-09-06T16:17:28.202Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.208Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2d2fe9b7c4c96ab66de4820a: done: 0.022721 sec","job_status":"done","queue_duration_s":2.094725,"scheduling_latency_s":2.094739,"redis_calls":10,"redis_duration_s":0.001619,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000315,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000307,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000282158,"duration_s":0.022721,"completed_at":"2026-09-06T16:17:30.325Z","load_balancing_strategy":"primary","db_duration_s":0.003393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.372625] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.327Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f87dd5f671ef1790acef1363","created_at":"2026-09-06T16:17:28.221Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.225Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f87dd5f671ef1790acef1363: start","job_status":"start","queue_duration_s":2.10207,"scheduling_latency_s":2.102084} gitlab # [ 261.416298] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.354Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f87dd5f671ef1790acef1363","created_at":"2026-09-06T16:17:28.221Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.225Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f87dd5f671ef1790acef1363: done: 0.027274 sec","job_status":"done","queue_duration_s":2.10207,"scheduling_latency_s":2.102084,"redis_calls":13,"redis_duration_s":0.002365,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000616,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000256458,"duration_s":0.027274,"completed_at":"2026-09-06T16:17:30.354Z","load_balancing_strategy":"primary","db_duration_s":0.004521,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.462175] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.355Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7970bf3c8bc68b13e9713b15","created_at":"2026-09-06T16:17:28.225Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.228Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7970bf3c8bc68b13e9713b15: start","job_status":"start","queue_duration_s":2.126896,"scheduling_latency_s":2.126917} gitlab # [ 261.462985] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.377Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7970bf3c8bc68b13e9713b15","created_at":"2026-09-06T16:17:28.225Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.228Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7970bf3c8bc68b13e9713b15: done: 0.02211 sec","job_status":"done","queue_duration_s":2.126896,"scheduling_latency_s":2.126917,"redis_calls":10,"redis_duration_s":0.001692,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000228241,"duration_s":0.02211,"completed_at":"2026-09-06T16:17:30.377Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003268,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.504064] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.379Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1919d7991bd709c1445ee174","created_at":"2026-09-06T16:17:28.229Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.321Z","job_size_bytes":19,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1919d7991bd709c1445ee174: start","job_status":"start","queue_duration_s":2.058182,"scheduling_latency_s":2.058197} gitlab # [ 261.504696] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.403Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1919d7991bd709c1445ee174","created_at":"2026-09-06T16:17:28.229Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.321Z","job_size_bytes":19,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1919d7991bd709c1445ee174: done: 0.02436 sec","job_status":"done","queue_duration_s":2.058182,"scheduling_latency_s":2.058197,"redis_calls":11,"redis_duration_s":0.002509,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001618,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000275733,"duration_s":0.02436,"completed_at":"2026-09-06T16:17:30.403Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.555068] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.405Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8a58f711285c4f56d7af8d36","created_at":"2026-09-06T16:17:28.323Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.327Z","job_size_bytes":66,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8a58f711285c4f56d7af8d36: start","job_status":"start","queue_duration_s":2.077729,"scheduling_latency_s":2.077744} gitlab # [ 261.555802] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.447Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8a58f711285c4f56d7af8d36","created_at":"2026-09-06T16:17:28.323Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.327Z","job_size_bytes":66,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8a58f711285c4f56d7af8d36: done: 0.042534 sec","job_status":"done","queue_duration_s":2.077729,"scheduling_latency_s":2.077744,"redis_calls":10,"redis_duration_s":0.002435,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000515,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337193,"duration_s":0.042534,"completed_at":"2026-09-06T16:17:30.447Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.595139] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.449Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f10003c613c88527678938b8","created_at":"2026-09-06T16:17:28.328Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.331Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f10003c613c88527678938b8: start","job_status":"start","queue_duration_s":2.117412,"scheduling_latency_s":2.117427} gitlab # [ 261.605498] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.477Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f10003c613c88527678938b8","created_at":"2026-09-06T16:17:28.328Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.331Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f10003c613c88527678938b8: done: 0.028437 sec","job_status":"done","queue_duration_s":2.117412,"scheduling_latency_s":2.117427,"redis_calls":10,"redis_duration_s":0.0019590000000000002,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000274336,"duration_s":0.028437,"completed_at":"2026-09-06T16:17:30.477Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004942,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.647892] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.478Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ee54884d0ddcda4829988e9d","created_at":"2026-09-06T16:17:28.336Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.339Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ee54884d0ddcda4829988e9d: start","job_status":"start","queue_duration_s":2.139187,"scheduling_latency_s":2.139202} gitlab # [ 261.660806] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.506Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ee54884d0ddcda4829988e9d","created_at":"2026-09-06T16:17:28.336Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.339Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ee54884d0ddcda4829988e9d: done: 0.027889 sec","job_status":"done","queue_duration_s":2.139187,"scheduling_latency_s":2.139202,"redis_calls":10,"redis_duration_s":0.002071,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000316241,"duration_s":0.027889,"completed_at":"2026-09-06T16:17:30.506Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.722916] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.507Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"652c73775ab1f6ae81def24f","created_at":"2026-09-06T16:17:28.339Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.342Z","job_size_bytes":6,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-652c73775ab1f6ae81def24f: start","job_status":"start","queue_duration_s":2.165161,"scheduling_latency_s":2.165176} gitlab # [ 261.723586] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"652c73775ab1f6ae81def24f","created_at":"2026-09-06T16:17:28.339Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.342Z","job_size_bytes":6,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-652c73775ab1f6ae81def24f: done: 0.10007 sec","job_status":"done","queue_duration_s":2.165161,"scheduling_latency_s":2.165176,"redis_calls":10,"redis_duration_s":0.081864,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000304,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.080498,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028132,"duration_s":0.10007,"completed_at":"2026-09-06T16:17:30.607Z","load_balancing_strategy":"primary","db_duration_s":0.002151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.765713] gitaly[927]: time="2026-09-06T16:17:30.742Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH diskcache=40c8d343-4449-444b-855b-373185cf1681 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:17:30.732" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:30.732" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 261.766501] gitaly[927]: time="2026-09-06T16:17:30.742Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:17:30.732" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:30.732" grpc.time_ms=9.305 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 261.808516] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.609Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"ff6b5da9080a3f265bd32c88","created_at":"2026-09-06T16:17:28.343Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.346Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-ff6b5da9080a3f265bd32c88: start","job_status":"start","queue_duration_s":2.263037,"scheduling_latency_s":2.263053} gitlab # [ 261.809285] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.634Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"ff6b5da9080a3f265bd32c88","created_at":"2026-09-06T16:17:28.343Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.346Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-ff6b5da9080a3f265bd32c88: done: 0.025121 sec","job_status":"done","queue_duration_s":2.263037,"scheduling_latency_s":2.263053,"redis_calls":10,"redis_duration_s":0.001407,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212597,"duration_s":0.025121,"completed_at":"2026-09-06T16:17:30.634Z","load_balancing_strategy":"primary","db_duration_s":0.004766,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.853733] gitaly[927]: time="2026-09-06T16:17:30.837Z" level=info msg="finished unary call with code OK" catfile.duration_ms=59 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=59 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:00.776" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:30.776" grpc.time_ms=60.194 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 261.854910] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.635Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"e7eddcf1593a8d70d6490a26","created_at":"2026-09-06T16:17:28.350Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.353Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-e7eddcf1593a8d70d6490a26: start","job_status":"start","queue_duration_s":2.281285,"scheduling_latency_s":2.281301} gitlab # [ 261.855611] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.680Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"e7eddcf1593a8d70d6490a26","created_at":"2026-09-06T16:17:28.350Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.353Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-e7eddcf1593a8d70d6490a26: done: 0.045688 sec","job_status":"done","queue_duration_s":2.281285,"scheduling_latency_s":2.281301,"redis_calls":14,"redis_duration_s":0.002116,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000257295,"duration_s":0.045688,"completed_at":"2026-09-06T16:17:30.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.894037] gitaly[927]: time="2026-09-06T16:17:30.847Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:17:40.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:30.840" grpc.time_ms=7.049 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 261.903461] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.686Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6b64f7e8da667946448af9ea","created_at":"2026-09-06T16:17:28.444Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.445Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6b64f7e8da667946448af9ea: start","job_status":"start","queue_duration_s":2.240568,"scheduling_latency_s":2.240588} gitlab # [ 261.945723] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.718Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6b64f7e8da667946448af9ea","created_at":"2026-09-06T16:17:28.444Z","trace_propagation_headers":{"sentry-trace":"f5e4133b95d442b9a0669f30f9005e66-362f5cec807d480a","baggage":"sentry-trace_id=f5e4133b95d442b9a0669f30f9005e66,sentry-release=Unknown"},"correlation_id":"01M1VR3WWGGZG02XKX3MSRR92Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.445Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6b64f7e8da667946448af9ea: done: 0.032159 sec","job_status":"done","queue_duration_s":2.240568,"scheduling_latency_s":2.240588,"redis_calls":8,"redis_duration_s":0.006158,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004318,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00184,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032159,"completed_at":"2026-09-06T16:17:30.718Z","load_balancing_strategy":"primary","db_duration_s":0.007851,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 261.954719] gitaly[927]: time="2026-09-06T16:17:30.873Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:18:00.872" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:30.872" grpc.time_ms=0.621 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 261.995689] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.719Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1f89af5662fd975026f0ce40","created_at":"2026-09-06T16:17:28.570Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-6319f31077c845e3","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"ba837407530a9d8a233800b4","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.572Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1f89af5662fd975026f0ce40: start","job_status":"start","queue_duration_s":2.14723,"scheduling_latency_s":2.14725} gitlab # [ 261.996351] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.757Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1f89af5662fd975026f0ce40","created_at":"2026-09-06T16:17:28.570Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-6319f31077c845e3","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"ba837407530a9d8a233800b4","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.572Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1f89af5662fd975026f0ce40: done: 0.037423 sec","job_status":"done","queue_duration_s":2.14723,"scheduling_latency_s":2.14725,"gitaly_calls":1,"gitaly_duration_s":0.011639,"redis_calls":12,"redis_duration_s":0.0018629999999999999,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000345,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000801,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.023740168,"duration_s":0.037423,"completed_at":"2026-09-06T16:17:30.757Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.038163] gitaly[927]: time="2026-09-06T16:17:30.917Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=30 command.oublock=0 command.real_time_ms=41 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:18:00.875" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:30.875" grpc.time_ms=41.896 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.038912] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:30.760Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"db22edb5e3762ee9c0c11deb","created_at":"2026-09-06T16:17:28.687Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-6319f31077c845e3","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3c171cc3c24d64e69ff41a6a68ec25dfdbdd89b751c7714549d12a2aad0894bc","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.689Z","job_size_bytes":467,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-db22edb5e3762ee9c0c11deb: start","job_status":"start","queue_duration_s":2.070942,"scheduling_latency_s":2.070961} gitlab # [ 262.082351] gitaly[927]: time="2026-09-06T16:17:30.946Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:40.110" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:30.940" grpc.time_ms=6.223 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.100686] gitlab-workhorse[824]: time="2026-09-06T16:17:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 262.103051] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 928 gitlab # [ 262.162237] gitaly[927]: time="2026-09-06T16:17:31.195Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EG0HNM5CKGC38SFBYEB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.182" pid=927 remote_ip="2001:db8:1::1" request_sha=da50364f0053f1a81391709a9143464f8cd0bb16 response_bytes=337 span.kind=server system=grpc gitlab # [ 262.172189] gitaly[927]: time="2026-09-06T16:17:31.196Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EG0HNM5CKGC38SFBYEB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.182" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 262.186181] gitlab-workhorse[824]: time="2026-09-06T16:17:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 262.186868] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 59 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d' gitlab # [ 262.187336] gitaly[927]: time="2026-09-06T16:17:31.196Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EG0HNM5CKGC38SFBYEB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.182" grpc.time_ms=13.708 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 262.217028] nixbot[2698]: {"ts": "2026-09-06T16:17:31.250Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 262.241928] nixbot[2698]: INFO: ::ffff:127.0.0.1:59554 - "GET /api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d', in 0.04 seconds) gitlab # [ 262.283649] gitaly[927]: time="2026-09-06T16:17:31.317Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EM0ATE07WWZW760VT5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:01.315" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:31.315" grpc.time_ms=0.952 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.332738] gitaly[927]: time="2026-09-06T16:17:31.366Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=970 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:18:01.355" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:31.356" grpc.time_ms=10.162 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.356381] gitaly[927]: time="2026-09-06T16:17:31.390Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR57ZCQDXKYF5GHC5KDVGH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:18:01.381" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:31.381" grpc.time_ms=8.582 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.369987] sidekiq-cluster[2565]: {"severity":"WARN","time":"2026-09-06T16:17:31.403Z","class":"CreatePipelineWorker","correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"1f01eb39b91f0d35d859c09f4fef763afded0209","after":"15ca3fb256fc690a0d60f7cdef964df568aa183d","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"15ca3fb256fc690a0d60f7cdef964df568aa183d","ref":"refs/heads/master"},"retry":0} gitlab # [ 262.376189] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.409Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"db22edb5e3762ee9c0c11deb","created_at":"2026-09-06T16:17:28.687Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-6319f31077c845e3","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3c171cc3c24d64e69ff41a6a68ec25dfdbdd89b751c7714549d12a2aad0894bc","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.689Z","job_size_bytes":467,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-db22edb5e3762ee9c0c11deb: done: 0.649214 sec","job_status":"done","queue_duration_s":2.070942,"scheduling_latency_s":2.070961,"gitaly_calls":7,"gitaly_duration_s":0.162809,"redis_calls":33,"redis_duration_s":0.004478,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000474,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002106,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.469871,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.649214,"completed_at":"2026-09-06T16:17:31.409Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013232,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 262.380866] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.413Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e3017a8c-4652-40c6-ba04-60d96965e153\"}"],"class":"WebHookWorker","jid":"c6b4593ac9b01d48c1227549","created_at":"2026-09-06T16:17:28.701Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-6319f31077c845e3","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b11692a1b176918c159dde9f9b94c6382269cd51d52bb5c5bdf1a9ad1cf84a03","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.702Z","job_size_bytes":1749,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c6b4593ac9b01d48c1227549: start","job_status":"start","queue_duration_s":2.711245,"scheduling_latency_s":2.711271} gitlab # [ 262.412212] nixbot[2698]: INFO: ::ffff:127.0.0.1:59570 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 262.421884] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.455Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e3017a8c-4652-40c6-ba04-60d96965e153\"}"],"class":"WebHookWorker","jid":"c6b4593ac9b01d48c1227549","created_at":"2026-09-06T16:17:28.701Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-6319f31077c845e3","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b11692a1b176918c159dde9f9b94c6382269cd51d52bb5c5bdf1a9ad1cf84a03","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.702Z","job_size_bytes":1749,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c6b4593ac9b01d48c1227549: done: 0.041576 sec","job_status":"done","queue_duration_s":2.711245,"scheduling_latency_s":2.711271,"redis_calls":14,"redis_duration_s":0.0046559999999999995,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000404,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003499,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000753,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008639646000005996,"cpu_s":0.019394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.041576,"completed_at":"2026-09-06T16:17:31.455Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004668,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.466053] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.460Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"b047c8eea32a7f50251f4100","created_at":"2026-09-06T16:17:28.821Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-2d52c613364443c5","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.825Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-b047c8eea32a7f50251f4100: start","job_status":"start","queue_duration_s":2.635097,"scheduling_latency_s":2.635116} gitlab # [ 262.466694] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.499Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"b047c8eea32a7f50251f4100","created_at":"2026-09-06T16:17:28.821Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-2d52c613364443c5","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:28.825Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-b047c8eea32a7f50251f4100: done: 0.038622 sec","job_status":"done","queue_duration_s":2.635097,"scheduling_latency_s":2.635116,"redis_calls":19,"redis_duration_s":0.006366999999999999,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002744,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.001599,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001506,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009250617,"duration_s":0.038622,"completed_at":"2026-09-06T16:17:31.499Z","load_balancing_strategy":"primary","db_duration_s":0.005604,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.511984] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.505Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","fb719291c989122e08e10eba"],"class":"WebHooks::LogExecutionWorker","jid":"59f0d0c7ff74b68ead4c66f2","created_at":"2026-09-06T16:17:30.079Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-7858931887ff4a07","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f9603dac7420995d1ae395d61f644ea16b5300bdc92562890fe0d4d1ccaacca9","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:30.088Z","job_size_bytes":2593,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-59f0d0c7ff74b68ead4c66f2: start","job_status":"start","queue_duration_s":1.416748,"scheduling_latency_s":1.416764} gitlab # [ 262.512795] gitaly[927]: time="2026-09-06T16:17:31.511Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5EM0ATE07WWZW760VT5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:41.012" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:31.502" grpc.time_ms=8.625 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.529195] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.562Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","fb719291c989122e08e10eba"],"class":"WebHooks::LogExecutionWorker","jid":"59f0d0c7ff74b68ead4c66f2","created_at":"2026-09-06T16:17:30.079Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-7858931887ff4a07","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f9603dac7420995d1ae395d61f644ea16b5300bdc92562890fe0d4d1ccaacca9","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:30.088Z","job_size_bytes":2593,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-59f0d0c7ff74b68ead4c66f2: done: 0.056774 sec","job_status":"done","queue_duration_s":1.416748,"scheduling_latency_s":1.416764,"redis_calls":10,"redis_duration_s":0.008464,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002981,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003904,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000477435,"duration_s":0.056774,"completed_at":"2026-09-06T16:17:31.562Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013653,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.599850] gitaly[927]: time="2026-09-06T16:17:31.565Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5EVRHKR8ZZATQ21QQ9EW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.556" grpc.time_ms=8.46 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 262.600756] nixbot[2698]: {"ts": "2026-09-06T16:17:31.575Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 \"HTTP/1.1 201 Created\""} gitlab # [ 262.601326] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.568Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c6b4593ac9b01d48c1227549"],"class":"WebHooks::LogExecutionWorker","jid":"af12fc7868c38f21649956cf","created_at":"2026-09-06T16:17:31.447Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-32cf92875dbe4d85","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c8fd4e67dd88f3dccf005008a6251b296dcc03bac7941ce468b63b9857bcc69b","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.450Z","job_size_bytes":2592,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-af12fc7868c38f21649956cf: start","job_status":"start","queue_duration_s":0.117904,"scheduling_latency_s":0.117925} gitlab # [ 262.601966] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.600Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c6b4593ac9b01d48c1227549"],"class":"WebHooks::LogExecutionWorker","jid":"af12fc7868c38f21649956cf","created_at":"2026-09-06T16:17:31.447Z","trace_propagation_headers":{"sentry-trace":"b9602346942a4626844d3e8438915014-32cf92875dbe4d85","baggage":"sentry-trace_id=b9602346942a4626844d3e8438915014,sentry-release=Unknown"},"correlation_id":"01M1VR57ZCQDXKYF5GHC5KDVGH","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c8fd4e67dd88f3dccf005008a6251b296dcc03bac7941ce468b63b9857bcc69b","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.450Z","job_size_bytes":2592,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-af12fc7868c38f21649956cf: done: 0.031749 sec","job_status":"done","queue_duration_s":0.117904,"scheduling_latency_s":0.117925,"redis_calls":7,"redis_duration_s":0.0048519999999999995,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003616,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031749,"completed_at":"2026-09-06T16:17:31.600Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.668306] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 52 gitlab # [ 262.669087] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 309 gitlab # [ 262.669583] gitlab-workhorse[824]: time="2026-09-06T16:17:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 262.671210] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 59 gitlab # [ 262.671893] gitaly[927]: time="2026-09-06T16:17:31.619Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EXWHF6P4ZBH54YCBNB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:01.618" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:31.619" grpc.time_ms=0.454 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.672612] gitaly[927]: time="2026-09-06T16:17:31.626Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EXF58ZKNQB2T7E8YAQA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.615" pid=927 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 262.673277] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.601Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"10f85021d9f4854287a323de","created_at":"2026-09-06T16:17:31.485Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-f31eff0f858246f4","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.491Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-10f85021d9f4854287a323de: start","job_status":"start","queue_duration_s":0.109915,"scheduling_latency_s":0.109932} gitlab # [ 262.716186] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.705Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"10f85021d9f4854287a323de","created_at":"2026-09-06T16:17:31.485Z","trace_propagation_headers":{"sentry-trace":"ebf9d83909784f5fbc672158d9c9024f-f31eff0f858246f4","baggage":"sentry-trace_id=ebf9d83909784f5fbc672158d9c9024f,sentry-release=Unknown"},"correlation_id":"01M1VR3RF27CH3HMPZEPBB39M5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.491Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-10f85021d9f4854287a323de: done: 0.103984 sec","job_status":"done","queue_duration_s":0.109915,"scheduling_latency_s":0.109932,"redis_calls":11,"redis_duration_s":0.007722,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001015,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005817,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000299479,"duration_s":0.103984,"completed_at":"2026-09-06T16:17:31.705Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.759819] gitaly[927]: time="2026-09-06T16:17:31.626Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EXF58ZKNQB2T7E8YAQA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.615" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 262.760721] nixbot[2698]: {"ts": "2026-09-06T16:17:31.760Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 \"HTTP/1.1 201 Created\""} gitlab # [ 262.761619] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.706Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7a23bbda4bccbc06a0b24517","created_at":"2026-09-06T16:17:31.550Z","trace_propagation_headers":{"sentry-trace":"6041bfd0a2624f5392b8b6480e87ffeb-d995a4f4cb5f4f80","baggage":"sentry-trace_id=6041bfd0a2624f5392b8b6480e87ffeb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5EM0ATE07WWZW760VT5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.554Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7a23bbda4bccbc06a0b24517: start","job_status":"start","queue_duration_s":0.152629,"scheduling_latency_s":0.15266} gitlab # [ 262.762254] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 81 gitlab # [ 262.762744] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 179 gitlab # [ 262.763250] gitaly[927]: time="2026-09-06T16:17:31.626Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EXF58ZKNQB2T7E8YAQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.615" grpc.time_ms=10.801 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 262.772658] gitaly[927]: time="2026-09-06T16:17:31.666Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5EXWHF6P4ZBH54YCBNB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:41.165" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:31.657" grpc.time_ms=8.449 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.785705] gitaly[927]: time="2026-09-06T16:17:31.740Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5F0BD16KM3GV69HA13XP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.731" grpc.time_ms=8.604 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 262.826767] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.817Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7a23bbda4bccbc06a0b24517","created_at":"2026-09-06T16:17:31.550Z","trace_propagation_headers":{"sentry-trace":"6041bfd0a2624f5392b8b6480e87ffeb-d995a4f4cb5f4f80","baggage":"sentry-trace_id=6041bfd0a2624f5392b8b6480e87ffeb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5EM0ATE07WWZW760VT5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.554Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2565,"job_status":"rescheduled","message":"PipelineProcessWorker JID-7a23bbda4bccbc06a0b24517: rescheduled"} gitlab # [ 262.827312] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.823Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7a23bbda4bccbc06a0b24517","created_at":"2026-09-06T16:17:31.550Z","trace_propagation_headers":{"sentry-trace":"6041bfd0a2624f5392b8b6480e87ffeb-d995a4f4cb5f4f80","baggage":"sentry-trace_id=6041bfd0a2624f5392b8b6480e87ffeb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5EM0ATE07WWZW760VT5S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.554Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7a23bbda4bccbc06a0b24517: done: 0.116368 sec","job_status":"done","queue_duration_s":0.152629,"scheduling_latency_s":0.15266,"gitaly_calls":2,"gitaly_duration_s":0.006042,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007489,"redis_read_bytes":59,"redis_write_bytes":6218,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000755,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000467,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_calls":4,"redis_queues_duration_s":0.000484,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2221,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001836,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":979,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00018,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000143,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002869,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":29,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":14,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078535,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.079095248,"duration_s":0.116368,"completed_at":"2026-09-06T16:17:31.823Z","load_balancing_strategy":"primary","db_duration_s":0.025249,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 262.884927] gitaly[927]: time="2026-09-06T16:17:31.787Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5EM0ATE07WWZW760VT5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:18:01.786" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:31.786" grpc.time_ms=0.74 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.885706] gitlab-workhorse[824]: time="2026-09-06T16:17:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 262.886151] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 97 gitlab # [ 262.886962] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.824Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f0b4731e352b3c4cfb444b95","created_at":"2026-09-06T16:17:31.772Z","trace_propagation_headers":{"sentry-trace":"6041bfd0a2624f5392b8b6480e87ffeb-dea3273a178848ef","baggage":"sentry-trace_id=6041bfd0a2624f5392b8b6480e87ffeb,sentry-release=Unknown"},"correlation_id":"01M1VR5EM0ATE07WWZW760VT5S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.774Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f0b4731e352b3c4cfb444b95: start","job_status":"start","queue_duration_s":0.05032,"scheduling_latency_s":0.05034} gitlab # [ 262.888975] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.856Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f0b4731e352b3c4cfb444b95","created_at":"2026-09-06T16:17:31.772Z","trace_propagation_headers":{"sentry-trace":"6041bfd0a2624f5392b8b6480e87ffeb-dea3273a178848ef","baggage":"sentry-trace_id=6041bfd0a2624f5392b8b6480e87ffeb,sentry-release=Unknown"},"correlation_id":"01M1VR5EM0ATE07WWZW760VT5S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.774Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f0b4731e352b3c4cfb444b95: done: 0.032177 sec","job_status":"done","queue_duration_s":0.05032,"scheduling_latency_s":0.05034,"redis_calls":7,"redis_duration_s":0.002729,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001993,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032177,"completed_at":"2026-09-06T16:17:31.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.930775] gitaly[927]: time="2026-09-06T16:17:31.810Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5EM0ATE07WWZW760VT5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:01.809" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:31.809" grpc.time_ms=0.629 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.936737] gitaly[927]: time="2026-09-06T16:17:31.836Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F2YCD7B2QRJ65FV1QNM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.829" pid=927 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 262.982168] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 64 gitlab # [ 262.982880] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 204 gitlab # [ 262.983541] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.859Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ee019e4cdb01a86da9a4d477","created_at":"2026-09-06T16:17:31.817Z","trace_propagation_headers":{"sentry-trace":"6041bfd0a2624f5392b8b6480e87ffeb-dea3273a178848ef","baggage":"sentry-trace_id=6041bfd0a2624f5392b8b6480e87ffeb,sentry-release=Unknown"},"correlation_id":"01M1VR5EM0ATE07WWZW760VT5S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.818Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ee019e4cdb01a86da9a4d477: start","job_status":"start","queue_duration_s":0.040468,"scheduling_latency_s":0.040499} gitlab # [ 262.984293] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.885Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ee019e4cdb01a86da9a4d477","created_at":"2026-09-06T16:17:31.817Z","trace_propagation_headers":{"sentry-trace":"6041bfd0a2624f5392b8b6480e87ffeb-dea3273a178848ef","baggage":"sentry-trace_id=6041bfd0a2624f5392b8b6480e87ffeb,sentry-release=Unknown"},"correlation_id":"01M1VR5EM0ATE07WWZW760VT5S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.818Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ee019e4cdb01a86da9a4d477: done: 0.026164 sec","job_status":"done","queue_duration_s":0.040468,"scheduling_latency_s":0.040499,"redis_calls":8,"redis_duration_s":0.001729,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001273,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026164,"completed_at":"2026-09-06T16:17:31.885Z","load_balancing_strategy":"primary","db_duration_s":0.003024,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.027160] gitaly[927]: time="2026-09-06T16:17:31.837Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F2YCD7B2QRJ65FV1QNM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.829" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 263.027848] nixbot[2698]: {"ts": "2026-09-06T16:17:31.969Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1f01eb39b91f0d35d859c09f4fef763afded0209 \"HTTP/1.1 201 Created\""} gitlab # [ 263.028482] gitlab-workhorse[824]: time="2026-09-06T16:17:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 263.028918] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 70 gitlab # [ 263.029796] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:31.946Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"792de28be39316b6a89aa35d","created_at":"2026-09-06T16:17:31.941Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-a48de8189b314111","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.945Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-792de28be39316b6a89aa35d: start","job_status":"start","queue_duration_s":0.001361,"scheduling_latency_s":0.001397} gitlab # [ 263.031595] gitaly[927]: time="2026-09-06T16:17:31.837Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F2YCD7B2QRJ65FV1QNM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.829" grpc.time_ms=7.664 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 263.076646] gitaly[927]: time="2026-09-06T16:17:31.845Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:01.844" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:31.845" grpc.time_ms=0.489 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.077283] gitaly[927]: time="2026-09-06T16:17:31.880Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:41.084" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:31.874" grpc.time_ms=6.45 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.077902] gitaly[927]: time="2026-09-06T16:17:31.948Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5F7BDSQ490WNER2S2MYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:31.941" grpc.time_ms=6.444 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 263.079238] gitaly[927]: time="2026-09-06T16:17:32.021Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F9F1APNAF4WNWY3J8ST grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:32.006" pid=927 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 263.088353] gitaly[927]: time="2026-09-06T16:17:32.021Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F9F1APNAF4WNWY3J8ST grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:32.006" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 263.131879] gitaly[927]: time="2026-09-06T16:17:32.021Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F9F1APNAF4WNWY3J8ST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:32.006" grpc.time_ms=15.166 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 263.133096] gitaly[927]: time="2026-09-06T16:17:32.049Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:32.048" grpc.time_ms=1.023 pid=927 span.kind=server system=grpc gitlab # [ 263.133821] gitaly[927]: time="2026-09-06T16:17:32.082Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:32.081" grpc.time_ms=0.45 pid=927 span.kind=server system=grpc gitlab # [ 263.134482] gitaly[927]: time="2026-09-06T16:17:32.101Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:32.101" grpc.time_ms=0.37 pid=927 span.kind=server system=grpc gitlab # [ 263.141671] gitaly[927]: time="2026-09-06T16:17:32.102Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:02.101" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:32.101" grpc.time_ms=0.904 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.188383] gitaly[927]: time="2026-09-06T16:17:32.108Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M diskcache=1c3b57c8-1055-40bb-b460-f3cdd8dd5902 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:18:02.029" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:32.029" pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.189131] gitaly[927]: time="2026-09-06T16:17:32.108Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7332 command.oublock=0 command.real_time_ms=73 command.spawn_token_fork_ms=0 command.system_time_ms=37 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:18:02.029" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:32.029" grpc.time_ms=78.8 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d' gitlab # [ 263.189789] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.217Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"792de28be39316b6a89aa35d","created_at":"2026-09-06T16:17:31.941Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-a48de8189b314111","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:31.945Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-792de28be39316b6a89aa35d: done: 0.270886 sec","job_status":"done","queue_duration_s":0.001361,"scheduling_latency_s":0.001397,"gitaly_calls":3,"gitaly_duration_s":0.087505,"redis_calls":71,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014275999999999999,"redis_read_bytes":151,"redis_write_bytes":19875,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000833,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002665,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":23,"redis_queues_duration_s":0.002568,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13377,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004802,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000196,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000396,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001983,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.123632,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.232131738,"duration_s":0.270886,"completed_at":"2026-09-06T16:17:32.217Z","load_balancing_strategy":"primary","db_duration_s":0.040194,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.240806] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.219Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"359823eff0ec6fef78ef7cde","created_at":"2026-09-06T16:17:32.021Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.024Z","job_size_bytes":23,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-359823eff0ec6fef78ef7cde: start","job_status":"start","queue_duration_s":0.194851,"scheduling_latency_s":0.194875} gitlab # [ 263.241627] gitaly[927]: time="2026-09-06T16:17:32.116Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:18:02.108" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:32.108" grpc.time_ms=8.039 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.286235] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.257Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"359823eff0ec6fef78ef7cde","created_at":"2026-09-06T16:17:32.021Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.024Z","job_size_bytes":23,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-359823eff0ec6fef78ef7cde: done: 0.03846 sec","job_status":"done","queue_duration_s":0.194851,"scheduling_latency_s":0.194875,"redis_calls":7,"redis_duration_s":0.001131,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03846,"completed_at":"2026-09-06T16:17:32.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008614,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.290797] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.258Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"11e36d523db85dd77c46fa08","created_at":"2026-09-06T16:17:32.114Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.117Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-11e36d523db85dd77c46fa08: start","job_status":"start","queue_duration_s":0.141273,"scheduling_latency_s":0.141297} gitlab # [ 263.333885] gitaly[927]: time="2026-09-06T16:17:32.127Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:42.082" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:32.122" grpc.time_ms=5.235 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.334657] nixbot[2698]: {"ts": "2026-09-06T16:17:32.354Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/15ca3fb256fc690a0d60f7cdef964df568aa183d \"HTTP/1.1 201 Created\""} gitlab # [ 263.335128] nixbot[2698]: {"ts": "2026-09-06T16:17:32.364Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-15ca3fb256fc-5"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d', in 0.15 seconds) (finished: subtest: push delivers the webhook, in 52.84 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses gitlab # [ 263.335574] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.282Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"11e36d523db85dd77c46fa08","created_at":"2026-09-06T16:17:32.114Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.117Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-11e36d523db85dd77c46fa08: done: 0.02348 sec","job_status":"done","queue_duration_s":0.141273,"scheduling_latency_s":0.141297,"redis_calls":7,"redis_duration_s":0.001372,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02348,"completed_at":"2026-09-06T16:17:32.282Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.360303] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.283Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6fc3aa17146ab9b0a16d7aad","created_at":"2026-09-06T16:17:32.123Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.126Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6fc3aa17146ab9b0a16d7aad: start","job_status":"start","queue_duration_s":0.156669,"scheduling_latency_s":0.15669} gitlab # [ 263.406093] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 301 gitlab # [ 263.406796] gitaly[927]: time="2026-09-06T16:17:32.144Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:17:42.103" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:32.143" grpc.time_ms=0.363 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.407455] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.304Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6fc3aa17146ab9b0a16d7aad","created_at":"2026-09-06T16:17:32.123Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.126Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6fc3aa17146ab9b0a16d7aad: done: 0.021537 sec","job_status":"done","queue_duration_s":0.156669,"scheduling_latency_s":0.15669,"redis_calls":7,"redis_duration_s":0.001091,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021537,"completed_at":"2026-09-06T16:17:32.304Z","load_balancing_strategy":"primary","db_duration_s":0.003272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.450784] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.305Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ca7c7400b9077bbbc297f3aa","created_at":"2026-09-06T16:17:32.133Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.135Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ca7c7400b9077bbbc297f3aa: start","job_status":"start","queue_duration_s":0.170594,"scheduling_latency_s":0.170638} gitlab # [ 263.451722] nixbot[2698]: INFO: ::ffff:127.0.0.1:35108 - "GET /api/repos/gitlab/root/test-flake/builds?commit=15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 200 OK gitlab # [ 263.452305] gitaly[927]: time="2026-09-06T16:17:32.152Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:17:42.106" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:32.146" grpc.time_ms=5.3 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses, in 0.10 seconds) gitlab # [ 263.452976] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.330Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ca7c7400b9077bbbc297f3aa","created_at":"2026-09-06T16:17:32.133Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.135Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ca7c7400b9077bbbc297f3aa: done: 0.025014 sec","job_status":"done","queue_duration_s":0.170594,"scheduling_latency_s":0.170638,"redis_calls":10,"redis_duration_s":0.002409,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00135,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025014,"completed_at":"2026-09-06T16:17:32.330Z","load_balancing_strategy":"primary","db_duration_s":0.004244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.500621] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"0120e1998d5d9961ef9b1f8f","created_at":"2026-09-06T16:17:32.135Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.137Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-0120e1998d5d9961ef9b1f8f: start","job_status":"start","queue_duration_s":0.194685,"scheduling_latency_s":0.194704} gitlab # [ 263.505505] gitaly[927]: time="2026-09-06T16:17:32.161Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:18:02.160" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:32.160" grpc.time_ms=0.54 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.554823] gitaly[927]: time="2026-09-06T16:17:32.184Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:32.184" grpc.time_ms=0.362 pid=927 span.kind=server system=grpc gitlab # [ 263.555681] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 74 gitlab # [ 263.556204] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.352Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"0120e1998d5d9961ef9b1f8f","created_at":"2026-09-06T16:17:32.135Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.137Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-0120e1998d5d9961ef9b1f8f: done: 0.019953 sec","job_status":"done","queue_duration_s":0.194685,"scheduling_latency_s":0.194704,"redis_calls":7,"redis_duration_s":0.001386,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019953,"completed_at":"2026-09-06T16:17:32.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.597734] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.355Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"772c5528f9dd0cfb1323ba39","created_at":"2026-09-06T16:17:32.137Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.139Z","job_size_bytes":19,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-772c5528f9dd0cfb1323ba39: start","job_status":"start","queue_duration_s":0.215831,"scheduling_latency_s":0.215854} gitlab # [ 263.598452] gitaly[927]: time="2026-09-06T16:17:32.204Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5F3MAXNEHXWEVWPEZ04M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:02.202" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:32.203" grpc.time_ms=0.749 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.599131] gitaly[927]: time="2026-09-06T16:17:32.229Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:17:32.228" grpc.time_ms=0.43 pid=927 span.kind=server system=grpc gitlab # [ 263.601739] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.380Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"772c5528f9dd0cfb1323ba39","created_at":"2026-09-06T16:17:32.137Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.139Z","job_size_bytes":19,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-772c5528f9dd0cfb1323ba39: done: 0.025143 sec","job_status":"done","queue_duration_s":0.215831,"scheduling_latency_s":0.215854,"redis_calls":8,"redis_duration_s":0.004071,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00316,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025143,"completed_at":"2026-09-06T16:17:32.380Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003046,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.640791] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.381Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"22b87d182deffb47333b4268","created_at":"2026-09-06T16:17:32.140Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.141Z","job_size_bytes":66,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-22b87d182deffb47333b4268: start","job_status":"start","queue_duration_s":0.239988,"scheduling_latency_s":0.240012} gitlab # [ 263.679283] gitaly[927]: time="2026-09-06T16:17:32.239Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S diskcache=dced4271-5a77-4b4f-b3b2-d14e6a97d18e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:17:42.113" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:32.153" pid=927 ref=refs/keep-around/15ca3fb256fc690a0d60f7cdef964df568aa183d remote_ip="2001:db8:1::1" revision=15ca3fb256fc690a0d60f7cdef964df568aa183d span.kind=server system=grpc username=root gitlab # [ 263.679945] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.398Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"22b87d182deffb47333b4268","created_at":"2026-09-06T16:17:32.140Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.141Z","job_size_bytes":66,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-22b87d182deffb47333b4268: done: 0.016966 sec","job_status":"done","queue_duration_s":0.239988,"scheduling_latency_s":0.240012,"redis_calls":7,"redis_duration_s":0.001009,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016966,"completed_at":"2026-09-06T16:17:32.398Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001984,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.724351] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.399Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"96992d6b5d2a578d01d28458","created_at":"2026-09-06T16:17:32.142Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.143Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-96992d6b5d2a578d01d28458: start","job_status":"start","queue_duration_s":0.255727,"scheduling_latency_s":0.255745} gitlab # [ 263.725112] gitaly[927]: time="2026-09-06T16:17:32.239Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=44 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5231 command.oublock=8 command.real_time_ms=79 command.spawn_token_fork_ms=0 command.system_time_ms=28 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:17:42.113" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:17:32.153" grpc.time_ms=85.171 pid=927 ref=refs/keep-around/15ca3fb256fc690a0d60f7cdef964df568aa183d remote_ip="2001:db8:1::1" revision=15ca3fb256fc690a0d60f7cdef964df568aa183d span.kind=server system=grpc username=root gitlab # [ 263.727073] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.432Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"96992d6b5d2a578d01d28458","created_at":"2026-09-06T16:17:32.142Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.143Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-96992d6b5d2a578d01d28458: done: 0.033241 sec","job_status":"done","queue_duration_s":0.255727,"scheduling_latency_s":0.255745,"redis_calls":7,"redis_duration_s":0.001399,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033241,"completed_at":"2026-09-06T16:17:32.432Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.774593] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.434Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d3c2e874003e128b6d8bdc2d","created_at":"2026-09-06T16:17:32.147Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.148Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d3c2e874003e128b6d8bdc2d: start","job_status":"start","queue_duration_s":0.286098,"scheduling_latency_s":0.286123} gitlab # [ 263.812078] gitaly[927]: time="2026-09-06T16:17:32.243Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:02.242" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:32.242" grpc.time_ms=0.291 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.812826] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.455Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d3c2e874003e128b6d8bdc2d","created_at":"2026-09-06T16:17:32.147Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.148Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d3c2e874003e128b6d8bdc2d: done: 0.021264 sec","job_status":"done","queue_duration_s":0.286098,"scheduling_latency_s":0.286123,"redis_calls":7,"redis_duration_s":0.001013,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021264,"completed_at":"2026-09-06T16:17:32.455Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.824906] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.456Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e1356fd2edc591dbb13490dc","created_at":"2026-09-06T16:17:32.148Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.149Z","job_size_bytes":6,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e1356fd2edc591dbb13490dc: start","job_status":"start","queue_duration_s":0.30693,"scheduling_latency_s":0.306945} gitlab # [ 263.865233] gitaly[927]: time="2026-09-06T16:17:32.497Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5FS1AQJP5AB9WXV9VS7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:02.196" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:32.496" grpc.time_ms=0.733 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.865704] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e1356fd2edc591dbb13490dc","created_at":"2026-09-06T16:17:32.148Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.149Z","job_size_bytes":6,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e1356fd2edc591dbb13490dc: done: 0.009703 sec","job_status":"done","queue_duration_s":0.30693,"scheduling_latency_s":0.306945,"redis_calls":7,"redis_duration_s":0.001191,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009703,"completed_at":"2026-09-06T16:17:32.466Z","load_balancing_strategy":"primary","db_duration_s":0.00029,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.885900] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.467Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"3c615f98778343098cb57aed","created_at":"2026-09-06T16:17:32.150Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.151Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3c615f98778343098cb57aed: start","job_status":"start","queue_duration_s":0.316233,"scheduling_latency_s":0.316248} gitlab # [ 263.932878] gitaly[927]: time="2026-09-06T16:17:32.619Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:18:02.617" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:32.617" grpc.time_ms=1.292 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.934068] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.483Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"3c615f98778343098cb57aed","created_at":"2026-09-06T16:17:32.150Z","trace_propagation_headers":{"sentry-trace":"ce94e43e475e444195304ef9760fd47e-549f2102bd37445d","baggage":"sentry-trace_id=ce94e43e475e444195304ef9760fd47e,sentry-release=Unknown"},"correlation_id":"01M1VR5F3MAXNEHXWEVWPEZ04M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.151Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3c615f98778343098cb57aed: done: 0.016593 sec","job_status":"done","queue_duration_s":0.316233,"scheduling_latency_s":0.316248,"redis_calls":7,"redis_duration_s":0.001675,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016593,"completed_at":"2026-09-06T16:17:32.483Z","load_balancing_strategy":"primary","db_duration_s":0.002401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.976216] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.484Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5f12a131681ea5cf4cf040bc","created_at":"2026-09-06T16:17:32.244Z","trace_propagation_headers":{"sentry-trace":"147c8486128148f98246f57732d913c7-ddff85e973794f7f","baggage":"sentry-trace_id=147c8486128148f98246f57732d913c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5FCMJ0952JDDG06SHE6S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.247Z","job_size_bytes":65,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5f12a131681ea5cf4cf040bc: start","job_status":"start","queue_duration_s":0.237624,"scheduling_latency_s":0.237639} gitlab # [ 263.976917] gitaly[927]: time="2026-09-06T16:17:32.651Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5FCMJ0952JDDG06SHE6S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:02.649" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:32.650" grpc.time_ms=0.828 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.977685] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.512Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5f12a131681ea5cf4cf040bc","created_at":"2026-09-06T16:17:32.244Z","trace_propagation_headers":{"sentry-trace":"147c8486128148f98246f57732d913c7-ddff85e973794f7f","baggage":"sentry-trace_id=147c8486128148f98246f57732d913c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5FCMJ0952JDDG06SHE6S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.247Z","job_size_bytes":65,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5f12a131681ea5cf4cf040bc: done: 0.027087 sec","job_status":"done","queue_duration_s":0.237624,"scheduling_latency_s":0.237639,"redis_calls":7,"redis_duration_s":0.00364,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027087,"completed_at":"2026-09-06T16:17:32.511Z","load_balancing_strategy":"primary","db_duration_s":0.004931,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.018790] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.513Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"43099c3578de0b10f22e8d46","created_at":"2026-09-06T16:17:32.248Z","trace_propagation_headers":{"sentry-trace":"147c8486128148f98246f57732d913c7-ddff85e973794f7f","baggage":"sentry-trace_id=147c8486128148f98246f57732d913c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5FCMJ0952JDDG06SHE6S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.249Z","job_size_bytes":65,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-43099c3578de0b10f22e8d46: start","job_status":"start","queue_duration_s":0.264245,"scheduling_latency_s":0.264272} gitlab # [ 264.022838] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.560Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"43099c3578de0b10f22e8d46","created_at":"2026-09-06T16:17:32.248Z","trace_propagation_headers":{"sentry-trace":"147c8486128148f98246f57732d913c7-ddff85e973794f7f","baggage":"sentry-trace_id=147c8486128148f98246f57732d913c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5FCMJ0952JDDG06SHE6S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.249Z","job_size_bytes":65,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-43099c3578de0b10f22e8d46: done: 0.046734 sec","job_status":"done","queue_duration_s":0.264245,"scheduling_latency_s":0.264272,"redis_calls":10,"redis_duration_s":0.001429,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000668,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046734,"completed_at":"2026-09-06T16:17:32.560Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.063088] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.561Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a046e975efebf5c0c7697c3a","created_at":"2026-09-06T16:17:32.338Z","trace_propagation_headers":{"sentry-trace":"147c8486128148f98246f57732d913c7-ddff85e973794f7f","baggage":"sentry-trace_id=147c8486128148f98246f57732d913c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5FCMJ0952JDDG06SHE6S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.341Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a046e975efebf5c0c7697c3a: start","job_status":"start","queue_duration_s":0.219975,"scheduling_latency_s":0.219995} gitlab # [ 264.074735] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.662Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a046e975efebf5c0c7697c3a","created_at":"2026-09-06T16:17:32.338Z","trace_propagation_headers":{"sentry-trace":"147c8486128148f98246f57732d913c7-ddff85e973794f7f","baggage":"sentry-trace_id=147c8486128148f98246f57732d913c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5FCMJ0952JDDG06SHE6S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.341Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a046e975efebf5c0c7697c3a: done: 0.100693 sec","job_status":"done","queue_duration_s":0.219975,"scheduling_latency_s":0.219995,"gitaly_calls":2,"gitaly_duration_s":0.007085,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006155,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000668,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00126,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000197,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001493,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000168,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000633,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068183,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.076182308,"duration_s":0.100693,"completed_at":"2026-09-06T16:17:32.662Z","load_balancing_strategy":"primary","db_duration_s":0.023422,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.121801] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.663Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"f044e766d9e7590e0237af19","created_at":"2026-09-06T16:17:32.603Z","trace_propagation_headers":{"sentry-trace":"147c8486128148f98246f57732d913c7-970dce645b43455f","baggage":"sentry-trace_id=147c8486128148f98246f57732d913c7,sentry-release=Unknown"},"correlation_id":"01M1VR5FCMJ0952JDDG06SHE6S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.605Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f044e766d9e7590e0237af19: start","job_status":"start","queue_duration_s":0.05779,"scheduling_latency_s":0.057809} gitlab # [ 264.159429] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:32.691Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"f044e766d9e7590e0237af19","created_at":"2026-09-06T16:17:32.603Z","trace_propagation_headers":{"sentry-trace":"147c8486128148f98246f57732d913c7-970dce645b43455f","baggage":"sentry-trace_id=147c8486128148f98246f57732d913c7,sentry-release=Unknown"},"correlation_id":"01M1VR5FCMJ0952JDDG06SHE6S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:32.605Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f044e766d9e7590e0237af19: done: 0.028257 sec","job_status":"done","queue_duration_s":0.05779,"scheduling_latency_s":0.057809,"redis_calls":7,"redis_duration_s":0.001523,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000293,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028257,"completed_at":"2026-09-06T16:17:32.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.230120] nixbot[2698]: {"ts": "2026-09-06T16:17:33.263Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-15ca3fb256fc-5", "argv0": "bwrap"} gitlab # [ 264.307670] nix-daemon[2858]: accepted connection from pid 3709, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses gitlab # [ 264.714593] gitaly[927]: time="2026-09-06T16:17:33.747Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5GW8JKXESWATQB434F45 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:03.446" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:33.747" grpc.time_ms=0.621 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.740122] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 196 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses, in 0.25 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses gitlab # [ 265.870702] gitaly[927]: time="2026-09-06T16:17:34.903Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5J2WH91VKMG71HMAV5X4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:04.602" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:34.902" grpc.time_ms=0.881 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.908166] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 128 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses, in 0.17 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses gitlab # [ 267.007842] gitaly[927]: time="2026-09-06T16:17:36.041Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5K77A9X4QBK5CVNNRDRC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:06.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:36.040" grpc.time_ms=0.606 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.049841] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 107 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses, in 0.14 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses gitlab # [ 268.152497] gitaly[927]: time="2026-09-06T16:17:37.185Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5MB3WFNHGSDPG7Q0W0RF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:07.183" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:37.184" grpc.time_ms=0.941 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 268.175364] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 84 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses, in 0.13 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses gitlab # [ 269.281216] gitaly[927]: time="2026-09-06T16:17:38.314Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5NEEZM2SSX704TM9KCRC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:08.312" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:38.313" grpc.time_ms=0.703 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.305939] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 83 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses, in 0.12 seconds) gitlab # [ 269.343972] nix-daemon[2858]: accepted connection from pid 3750, user nixbot gitlab # [ 269.358642] nix-daemon[2858]: accepted connection from pid 3709, user nixbot gitlab # [ 269.452830] gitaly[927]: time="2026-09-06T16:17:38.486Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5NM1190RDCJRTCKBWGHQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:08.484" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:38.485" grpc.time_ms=0.823 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.541606] gitaly[927]: time="2026-09-06T16:17:38.575Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5NM1190RDCJRTCKBWGHQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:48.074" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:38.565" grpc.time_ms=9.872 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.585465] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:38.618Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6b45bc1bd69d880ea7324057","created_at":"2026-09-06T16:17:38.599Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"986ed4d75986c77b833ac7ea078d5b60","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.614Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6b45bc1bd69d880ea7324057: start","job_status":"start","queue_duration_s":0.003409,"scheduling_latency_s":0.003457} gitlab # [ 269.614412] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 214 gitlab # [ 269.618596] nixbot[2698]: {"ts": "2026-09-06T16:17:38.651Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/15ca3fb256fc690a0d60f7cdef964df568aa183d \"HTTP/1.1 201 Created\""} gitlab # [ 269.640455] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:38.673Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6b45bc1bd69d880ea7324057","created_at":"2026-09-06T16:17:38.599Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"986ed4d75986c77b833ac7ea078d5b60","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.614Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6b45bc1bd69d880ea7324057: done: 0.055199 sec","job_status":"done","queue_duration_s":0.003409,"scheduling_latency_s":0.003457,"redis_calls":9,"redis_duration_s":0.005669,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001074,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002507,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002088,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055199,"completed_at":"2026-09-06T16:17:38.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.671604] gitaly[927]: time="2026-09-06T16:17:38.705Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5NTYRRC3B83CHVQ9MVRB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:08.703" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:38.704" grpc.time_ms=0.739 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.692407] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:38.725Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e00164b7acb0a9f6c101cab6","created_at":"2026-09-06T16:17:38.627Z","trace_propagation_headers":{"sentry-trace":"ab9a518161d3460d85efcceeed97efdd-27ba1a6138b749eb","baggage":"sentry-trace_id=ab9a518161d3460d85efcceeed97efdd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5NM1190RDCJRTCKBWGHQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.633Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e00164b7acb0a9f6c101cab6: start","job_status":"start","queue_duration_s":0.092766,"scheduling_latency_s":0.092801} gitlab # [ 269.718415] gitaly[927]: time="2026-09-06T16:17:38.752Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5NTYRRC3B83CHVQ9MVRB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:48.241" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:38.731" grpc.time_ms=20.487 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.774807] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 153 gitlab # [ 269.778231] nixbot[2698]: {"ts": "2026-09-06T16:17:38.811Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/15ca3fb256fc690a0d60f7cdef964df568aa183d \"HTTP/1.1 201 Created\""} gitlab # [ 269.852454] gitaly[927]: time="2026-09-06T16:17:38.886Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5P0CXA1RTPRWVZBG3NQB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:08.584" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:38.885" grpc.time_ms=0.583 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.881104] gitaly[927]: time="2026-09-06T16:17:38.914Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5NM1190RDCJRTCKBWGHQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:18:08.613" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:38.913" grpc.time_ms=0.917 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.925747] gitaly[927]: time="2026-09-06T16:17:38.959Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5NM1190RDCJRTCKBWGHQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:08.657" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:38.958" grpc.time_ms=0.653 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.960139] gitaly[927]: time="2026-09-06T16:17:38.993Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5P0CXA1RTPRWVZBG3NQB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:17:48.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:17:38.987" grpc.time_ms=6.612 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.960988] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:38.993Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e00164b7acb0a9f6c101cab6","created_at":"2026-09-06T16:17:38.627Z","trace_propagation_headers":{"sentry-trace":"ab9a518161d3460d85efcceeed97efdd-27ba1a6138b749eb","baggage":"sentry-trace_id=ab9a518161d3460d85efcceeed97efdd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5NM1190RDCJRTCKBWGHQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.633Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2565,"job_status":"rescheduled","message":"PipelineProcessWorker JID-e00164b7acb0a9f6c101cab6: rescheduled"} gitlab # [ 269.976222] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.009Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e00164b7acb0a9f6c101cab6","created_at":"2026-09-06T16:17:38.627Z","trace_propagation_headers":{"sentry-trace":"ab9a518161d3460d85efcceeed97efdd-27ba1a6138b749eb","baggage":"sentry-trace_id=ab9a518161d3460d85efcceeed97efdd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VR5NM1190RDCJRTCKBWGHQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.633Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e00164b7acb0a9f6c101cab6: done: 0.28396 sec","job_status":"done","queue_duration_s":0.092766,"scheduling_latency_s":0.092801,"gitaly_calls":2,"gitaly_duration_s":0.033027,"redis_calls":40,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025979,"redis_read_bytes":88,"redis_write_bytes":7682,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002219,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002423,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":6,"redis_queues_duration_s":0.003029,"redis_queues_read_bytes":7,"redis_queues_write_bytes":3278,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.007706,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1187,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.002066,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.002366,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003951,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":29,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.092,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":14,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.016,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.046,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.107255,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.172323577,"duration_s":0.28396,"completed_at":"2026-09-06T16:17:39.009Z","load_balancing_strategy":"primary","db_duration_s":0.10918,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 269.991587] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.012Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0ebf0585e3de6495a4918947","created_at":"2026-09-06T16:17:38.628Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"781ab2842a8d4cb595eb68bf21fbb3b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:17:38.646Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0ebf0585e3de6495a4918947: start","job_status":"start","queue_duration_s":0.366333,"scheduling_latency_s":0.366356} gitlab # [ 270.036775] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.026Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 270.037579] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0ebf0585e3de6495a4918947","created_at":"2026-09-06T16:17:38.628Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"781ab2842a8d4cb595eb68bf21fbb3b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:17:38.646Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0ebf0585e3de6495a4918947: done: 0.023286 sec","job_status":"done","queue_duration_s":0.366333,"scheduling_latency_s":0.366356,"redis_calls":9,"redis_duration_s":0.001367,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000368,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023286,"completed_at":"2026-09-06T16:17:39.035Z","load_balancing_strategy":"primary","db_duration_s":0.003146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.076563] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.038Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"82ccae6dc65570c3f3e3f7ce","created_at":"2026-09-06T16:17:38.658Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ceb4b4c25d71620cf0d8b03876761874","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.750Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-82ccae6dc65570c3f3e3f7ce: start","job_status":"start","queue_duration_s":0.287678,"scheduling_latency_s":0.287697} gitlab # [ 270.151494] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"82ccae6dc65570c3f3e3f7ce","created_at":"2026-09-06T16:17:38.658Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ceb4b4c25d71620cf0d8b03876761874","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.750Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-82ccae6dc65570c3f3e3f7ce: done: 0.146402 sec","job_status":"done","queue_duration_s":0.287678,"scheduling_latency_s":0.287697,"redis_calls":18,"redis_duration_s":0.015514,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.007495,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.001166,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.004361,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002492,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.146402,"completed_at":"2026-09-06T16:17:39.184Z","load_balancing_strategy":"primary","db_duration_s":0.00282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.159078] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9e3018ca84c5525238b77e5c","created_at":"2026-09-06T16:17:38.768Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"593072b995ab1f8f5457116971506bb7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.775Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9e3018ca84c5525238b77e5c: start","job_status":"start","queue_duration_s":0.416687,"scheduling_latency_s":0.416733} gitlab # [ 270.202994] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/15ca3fb256fc690a0d60f7cdef964df568aa183d HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 378 gitlab # [ 270.203780] nixbot[2698]: {"ts": "2026-09-06T16:17:39.208Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/15ca3fb256fc690a0d60f7cdef964df568aa183d \"HTTP/1.1 201 Created\""} gitlab # [ 270.266074] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.299Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9e3018ca84c5525238b77e5c","created_at":"2026-09-06T16:17:38.768Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"593072b995ab1f8f5457116971506bb7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.775Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9e3018ca84c5525238b77e5c: done: 0.107281 sec","job_status":"done","queue_duration_s":0.416687,"scheduling_latency_s":0.416733,"redis_calls":70,"redis_duration_s":0.018987,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002341,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000592,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.015391,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000663,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.107281,"completed_at":"2026-09-06T16:17:39.299Z","load_balancing_strategy":"primary","db_duration_s":0.002759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses gitlab # [ 270.305916] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.339Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"84f5094e7a032db43cef1e3a","created_at":"2026-09-06T16:17:38.794Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e708fa743a6c51d72cb244c5abf6e09","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.801Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-84f5094e7a032db43cef1e3a: start","job_status":"start","queue_duration_s":0.537531,"scheduling_latency_s":0.537575} gitlab # [ 270.318266] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.352Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 270.328667] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.361Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"84f5094e7a032db43cef1e3a","created_at":"2026-09-06T16:17:38.794Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e708fa743a6c51d72cb244c5abf6e09","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.801Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-84f5094e7a032db43cef1e3a: done: 0.022263 sec","job_status":"done","queue_duration_s":0.537531,"scheduling_latency_s":0.537575,"redis_calls":9,"redis_duration_s":0.0020959999999999998,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000394,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022263,"completed_at":"2026-09-06T16:17:39.361Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.330910] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.364Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a72c729b45a7fc7e6c3a9d26","created_at":"2026-09-06T16:17:38.813Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8be9c536304d4dc1e114a0763ffc84f6","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.821Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a72c729b45a7fc7e6c3a9d26: start","job_status":"start","queue_duration_s":0.54287,"scheduling_latency_s":0.542895} gitlab # [ 270.355933] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a72c729b45a7fc7e6c3a9d26","created_at":"2026-09-06T16:17:38.813Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8be9c536304d4dc1e114a0763ffc84f6","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.821Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a72c729b45a7fc7e6c3a9d26: done: 0.024898 sec","job_status":"done","queue_duration_s":0.54287,"scheduling_latency_s":0.542895,"redis_calls":11,"redis_duration_s":0.004069,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002644,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000433016,"exclusive_lock_hold_duration_s":0.002539429,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.024898,"completed_at":"2026-09-06T16:17:39.389Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.414694] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"be174d4b8c08c83dd4c6d89e","created_at":"2026-09-06T16:17:38.842Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41366c6d9fee90663edb90b042ad7ba4","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.861Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-be174d4b8c08c83dd4c6d89e: start","job_status":"start","queue_duration_s":0.534828,"scheduling_latency_s":0.534862} gitlab # [ 270.415307] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"be174d4b8c08c83dd4c6d89e","created_at":"2026-09-06T16:17:38.842Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41366c6d9fee90663edb90b042ad7ba4","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.861Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-be174d4b8c08c83dd4c6d89e: done: 0.02431 sec","job_status":"done","queue_duration_s":0.534828,"scheduling_latency_s":0.534862,"redis_calls":9,"redis_duration_s":0.002548,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000343,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001628,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02431,"completed_at":"2026-09-06T16:17:39.420Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002027,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.424870] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.422Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"66bd005d16acf3ab54d931c1","created_at":"2026-09-06T16:17:38.880Z","trace_propagation_headers":{"sentry-trace":"ab9a518161d3460d85efcceeed97efdd-08164a9064ee4536","baggage":"sentry-trace_id=ab9a518161d3460d85efcceeed97efdd,sentry-release=Unknown"},"correlation_id":"01M1VR5NM1190RDCJRTCKBWGHQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.882Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-66bd005d16acf3ab54d931c1: start","job_status":"start","queue_duration_s":0.539863,"scheduling_latency_s":0.539885} gitlab # [ 270.464737] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"66bd005d16acf3ab54d931c1","created_at":"2026-09-06T16:17:38.880Z","trace_propagation_headers":{"sentry-trace":"ab9a518161d3460d85efcceeed97efdd-08164a9064ee4536","baggage":"sentry-trace_id=ab9a518161d3460d85efcceeed97efdd,sentry-release=Unknown"},"correlation_id":"01M1VR5NM1190RDCJRTCKBWGHQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.882Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-66bd005d16acf3ab54d931c1: done: 0.043847 sec","job_status":"done","queue_duration_s":0.539863,"scheduling_latency_s":0.539885,"redis_calls":7,"redis_duration_s":0.0026780000000000003,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043847,"completed_at":"2026-09-06T16:17:39.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.502905] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8219cf3544bae7c178cda8b8","created_at":"2026-09-06T16:17:38.876Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cb9bfbf30815cf4b8003a18eedce40c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.885Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8219cf3544bae7c178cda8b8: start","job_status":"start","queue_duration_s":0.581889,"scheduling_latency_s":0.582487} gitlab # [ 270.503659] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.525Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8219cf3544bae7c178cda8b8","created_at":"2026-09-06T16:17:38.876Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cb9bfbf30815cf4b8003a18eedce40c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.885Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8219cf3544bae7c178cda8b8: done: 0.058351 sec","job_status":"done","queue_duration_s":0.581889,"scheduling_latency_s":0.582487,"redis_calls":11,"redis_duration_s":0.011897,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004051,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004616,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005984,"exclusive_lock_hold_duration_s":0.025595431,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.058351,"completed_at":"2026-09-06T16:17:39.525Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.559758] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.527Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"646fe464080c1c1044c61054","created_at":"2026-09-06T16:17:38.897Z","trace_propagation_headers":{"sentry-trace":"ab9a518161d3460d85efcceeed97efdd-08164a9064ee4536","baggage":"sentry-trace_id=ab9a518161d3460d85efcceeed97efdd,sentry-release=Unknown"},"correlation_id":"01M1VR5NM1190RDCJRTCKBWGHQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.899Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-646fe464080c1c1044c61054: start","job_status":"start","queue_duration_s":0.628302,"scheduling_latency_s":0.628356} gitlab # [ 270.560753] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.548Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"646fe464080c1c1044c61054","created_at":"2026-09-06T16:17:38.897Z","trace_propagation_headers":{"sentry-trace":"ab9a518161d3460d85efcceeed97efdd-08164a9064ee4536","baggage":"sentry-trace_id=ab9a518161d3460d85efcceeed97efdd,sentry-release=Unknown"},"correlation_id":"01M1VR5NM1190RDCJRTCKBWGHQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.899Z","job_size_bytes":3,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-646fe464080c1c1044c61054: done: 0.020531 sec","job_status":"done","queue_duration_s":0.628302,"scheduling_latency_s":0.628356,"redis_calls":7,"redis_duration_s":0.00164,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020531,"completed_at":"2026-09-06T16:17:39.548Z","load_balancing_strategy":"primary","db_duration_s":0.003215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.603698] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0b53db3860bd843b048c379e","created_at":"2026-09-06T16:17:38.895Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29ffcf4374d33639e041fc060a673970","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:17:38.900Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0b53db3860bd843b048c379e: start","job_status":"start","queue_duration_s":0.649106,"scheduling_latency_s":0.649124} gitlab # [ 270.605248] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 64 gitlab # [ 270.605878] gitaly[927]: time="2026-09-06T16:17:39.573Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5PNP4QXWFFFZ4XH7J166 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:39.567" grpc.time_ms=6.103 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 270.606325] gitaly[927]: time="2026-09-06T16:17:39.638Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5PQRR7AH0BH707DZ1589 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:39.627" pid=927 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 270.611500] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0b53db3860bd843b048c379e","created_at":"2026-09-06T16:17:38.895Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29ffcf4374d33639e041fc060a673970","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:17:38.900Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0b53db3860bd843b048c379e: done: 0.032519 sec","job_status":"done","queue_duration_s":0.649106,"scheduling_latency_s":0.649124,"redis_calls":9,"redis_duration_s":0.0035229999999999997,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000361,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000383,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032519,"completed_at":"2026-09-06T16:17:39.582Z","load_balancing_strategy":"primary","db_duration_s":0.003777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.683789] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.584Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"2d6c1719db5934ce573cdaf2","created_at":"2026-09-06T16:17:38.907Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52a980ffa753fc8121f999b75b5cdcf1","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.914Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-2d6c1719db5934ce573cdaf2: start","job_status":"start","queue_duration_s":0.670061,"scheduling_latency_s":0.67009} gitlab # [ 270.684750] gitlab-workhorse[824]: time="2026-09-06T16:17:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 270.685415] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 62 gitlab # [ 270.685905] gitaly[927]: time="2026-09-06T16:17:39.638Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5PQRR7AH0BH707DZ1589 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:39.627" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 270.689451] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.619Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"2d6c1719db5934ce573cdaf2","created_at":"2026-09-06T16:17:38.907Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52a980ffa753fc8121f999b75b5cdcf1","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.914Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-2d6c1719db5934ce573cdaf2: done: 0.035027 sec","job_status":"done","queue_duration_s":0.670061,"scheduling_latency_s":0.67009,"redis_calls":12,"redis_duration_s":0.005337,"redis_read_bytes":11,"redis_write_bytes":1354,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001837,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":297,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002396,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":519,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001268318,"exclusive_lock_hold_duration_s":0.008873195,"duration_s":0.035027,"completed_at":"2026-09-06T16:17:39.619Z","load_balancing_strategy":"primary","db_duration_s":0.005219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.713705] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"87ace2be5f79a1d5bbcaaf6d","created_at":"2026-09-06T16:17:38.918Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebe84851f23b1195a8bb132c38b7178d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.922Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-87ace2be5f79a1d5bbcaaf6d: start","job_status":"start","queue_duration_s":0.698671,"scheduling_latency_s":0.698692} gitlab # [ 270.767799] gitaly[927]: time="2026-09-06T16:17:39.638Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5PQRR7AH0BH707DZ1589 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:39.627" grpc.time_ms=10.882 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 270.768457] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/09/06:16:17:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 344 gitlab # [ 270.769114] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"87ace2be5f79a1d5bbcaaf6d","created_at":"2026-09-06T16:17:38.918Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebe84851f23b1195a8bb132c38b7178d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.922Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-87ace2be5f79a1d5bbcaaf6d: done: 0.040406 sec","job_status":"done","queue_duration_s":0.698671,"scheduling_latency_s":0.698692,"redis_calls":12,"redis_duration_s":0.004613,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000525,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009686706,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.040406,"completed_at":"2026-09-06T16:17:39.661Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006733,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 270.807423] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f0c39af7652893aa444494ec","created_at":"2026-09-06T16:17:38.925Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e8335f7675ba66b1c92e35fbdcab4ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.935Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f0c39af7652893aa444494ec: start","job_status":"start","queue_duration_s":0.729202,"scheduling_latency_s":0.729234} gitlab # [ 270.808192] gitaly[927]: time="2026-09-06T16:17:39.682Z" level=info msg="finished unary call with code OK" catfile.duration_ms=28 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=28 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VR5PKXSWR4M5B44Q3B9GJ2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:18:09.653" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:17:39.653" grpc.time_ms=28.779 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/15ca3fb256fc690a0d60f7cdef964df568aa183d/statuses, in 0.55 seconds) (finished: subtest: eval and build post commit statuses, in 7.47 seconds) (finished: run the VM test script, in 317.29 seconds) gitlab # [ 270.808810] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f0c39af7652893aa444494ec","created_at":"2026-09-06T16:17:38.925Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e8335f7675ba66b1c92e35fbdcab4ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.935Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f0c39af7652893aa444494ec: done: 0.119358 sec","job_status":"done","queue_duration_s":0.729202,"scheduling_latency_s":0.729234,"redis_calls":13,"redis_duration_s":0.00297,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000558,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00155,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.087,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000407314,"exclusive_lock_hold_duration_s":0.097316279,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.119358,"completed_at":"2026-09-06T16:17:39.783Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.088549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.886465] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.785Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"73e40f60074175c559379ef1","created_at":"2026-09-06T16:17:38.944Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"394dea80129a3d75eb317e6e88becc44","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.955Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-73e40f60074175c559379ef1: start","job_status":"start","queue_duration_s":0.829949,"scheduling_latency_s":0.829966} gitlab # [ 270.900531] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.794Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 270.903663] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"73e40f60074175c559379ef1","created_at":"2026-09-06T16:17:38.944Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"394dea80129a3d75eb317e6e88becc44","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.955Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-73e40f60074175c559379ef1: done: 0.014172 sec","job_status":"done","queue_duration_s":0.829949,"scheduling_latency_s":0.829966,"redis_calls":9,"redis_duration_s":0.0013800000000000002,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000308,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000335,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014172,"completed_at":"2026-09-06T16:17:39.799Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.942651] sidekiq-cluster[2565]: {"severity":"INFO","time":"2026-09-06T16:17:39.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"64187ef4e8c83a2201fe9750","created_at":"2026-09-06T16:17:38.976Z","trace_propagation_headers":{"sentry-trace":"cfe414c8365946eda353ba9d10c74d65-cc7098a57c4d49f9","baggage":"sentry-trace_id=cfe414c8365946eda353ba9d10c74d65,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92629947a827e9f6cc039f30420f246b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:17:38.985Z","job_size_bytes":2,"pid":2565,"sidekiq_tid":"m829","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-64187ef4e8c83a2201fe9750: start","job_status":"start","queue_duration_s":0.814512,"scheduling_latency_s":0.814527} test script finished in 317.39s cleanup gitlab # [ 270.956447] gitaly[927]: time="2026-09-06T16:17:39.890Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VR5PVEFMZBJ5P3BW688BF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:17:39.874" grpc.time_ms=16.181 pid=927 remote_ip="2001:db8:1::1" span.kind=server system=grpc kill QemuMachine (pid 46) gitlab # [ 270.958054] gitaly[927]: time="2026-09-06T16:17:39.932Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VR5NM1190RDCJRTCKBWGHQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=Refqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.59 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-06T16:16:38.963Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-06T16:17:32.046467Z', 'started_at': '2026-09-06T16:17:32.361322Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:17:32.331Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:17:32.331Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:17:32.331Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:17:32.331Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:17:32.331Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:17:32.331Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-09-06T16:17:32.331Z', 'started_at': None, 'finished_at': '2026-09-06T16:17:38.616Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '15ca3fb256fc690a0d60f7cdef964df568aa183d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-06T16:17:38.786Z', 'started_at': None, 'finished_at': '2026-09-06T16:17:39.163Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]