tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... (finished: start all VMs, in 48.91 seconds) gitlab # Formatting '/build/vm-state-gitlab/tmp.bX0PO7hjeg', fmt=raw size=1073741824 gitlab: waiting for unit setup-gitlab.service gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab: waiting for the VM to finish booting gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: e3d4486a-d114-4147-ad52-a3c4f7807bf9 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/njaigjkyls05gc5nhj73sfrdwxdhmf08-nixos-system-gitlab-test/init regInfo=/nix/store/y1hl3p14bhlf6ah7bwb0b97y94i7sil9-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 492126412 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor gitlab # [ 0.000809] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000878] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002734] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002745] Using GB pages for direct mapping gitlab # [ 0.002847] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002856] ACPI: Early table checksum verification disabled gitlab # [ 0.002859] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002863] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002867] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002874] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002876] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002878] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002880] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002881] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002882] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002883] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002884] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002884] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002885] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002885] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003127] No NUMA configuration found gitlab # [ 0.003128] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003130] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff] gitlab # [ 0.005663] Zone ranges: gitlab # [ 0.005663] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005665] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005666] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005667] Device empty gitlab # [ 0.005668] Movable zone start for each node gitlab # [ 0.005668] Early memory node ranges gitlab # [ 0.005669] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005670] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005671] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005672] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005693] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005977] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.122213] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.122685] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.122701] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.122736] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.122739] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.122741] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.122742] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.122743] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.122744] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.122747] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.122748] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.122754] TSC deadline timer available gitlab # [ 0.122758] CPU topo: Max. logical packages: 1 gitlab # [ 0.122759] CPU topo: Max. logical dies: 1 gitlab # [ 0.122759] CPU topo: Max. dies per package: 1 gitlab # [ 0.122762] CPU topo: Max. threads per core: 1 gitlab # [ 0.122763] CPU topo: Num. cores per package: 4 gitlab # [ 0.122764] CPU topo: Num. threads per package: 4 gitlab # [ 0.122764] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.122790] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.122808] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.122810] kvm-guest: setup PV sched yield gitlab # [ 0.122827] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.122829] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.122830] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.122831] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.122833] Booting paravirtualized kernel on KVM gitlab # [ 0.122837] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.127359] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.129726] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.129778] kvm-guest: PV spinlocks enabled gitlab # [ 0.129780] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.129783] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/njaigjkyls05gc5nhj73sfrdwxdhmf08-nixos-system-gitlab-test/init regInfo=/nix/store/y1hl3p14bhlf6ah7bwb0b97y94i7sil9-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.129882] Unknown kernel command line parameters "regInfo=/nix/store/y1hl3p14bhlf6ah7bwb0b97y94i7sil9-closure-info/registration", will be passed to user space. gitlab # [ 0.129895] random: crng init done gitlab # [ 0.129896] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.139371] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.144410] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.144487] software IO TLB: area num 4. gitlab # [ 0.223412] Fallback order for Node 0: 0 gitlab # [ 0.223420] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.223422] Policy zone: Normal gitlab # [ 0.225916] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.236299] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.250127] allocated 12582912 bytes of page_ext gitlab # [ 0.260300] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.260302] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.261184] Dynamic Preempt: lazy gitlab # [ 0.261324] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.261325] rcu: RCU event tracing is enabled. gitlab # [ 0.261326] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.261327] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.261328] Rude variant of Tasks RCU enabled. gitlab # [ 0.261328] Tracing variant of Tasks RCU enabled. gitlab # [ 0.261329] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.261329] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.261350] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.261352] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.261353] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.265740] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.266037] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.266046] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.266162] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.269730] Console: colour VGA+ 80x25 gitlab # [ 0.269733] printk: legacy console [tty0] enabled gitlab # [ 0.300715] printk: legacy console [ttyS0] enabled gitlab # [ 0.408853] ACPI: Core revision 20250807 gitlab # [ 0.409846] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.411474] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.412537] x2apic enabled gitlab # [ 0.413290] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.414238] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.415549] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.417052] kvm-guest: setup PV IPIs gitlab # [ 0.418703] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.419767] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.421598] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) gitlab # [ 0.422684] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.423745] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.425595] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.426599] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.428595] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.429595] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.430595] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.431595] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.432595] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.433604] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.434595] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.435595] active return thunk: srso_alias_return_thunk gitlab # [ 0.437532] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.438594] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.439593] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.440595] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.441593] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.442594] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.444461] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.445595] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.446595] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.447595] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.448593] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.449594] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.450594] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.481244] Freeing SMP alternatives memory: 44K gitlab # [ 0.481598] pid_max: default: 32768 minimum: 301 gitlab # [ 0.482712] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.483701] landlock: Up and running. gitlab # [ 0.484404] Yama: becoming mindful. gitlab # [ 0.485807] LSM support for eBPF active gitlab # [ 0.486825] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.487745] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.490915] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.492192] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.492604] ... version: 2 gitlab # [ 0.493391] ... bit width: 48 gitlab # [ 0.493597] ... generic counters: 6 gitlab # [ 0.494343] ... generic bitmap: 000000000000003f gitlab # [ 0.494597] ... fixed-purpose counters: 0 gitlab # [ 0.495336] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.495597] ... value mask: 0000ffffffffffff gitlab # [ 0.496565] ... max period: 00007fffffffffff gitlab # [ 0.497435] ... global_ctrl mask: 000000000000003f gitlab # [ 0.498518] signal: max sigframe size: 3376 gitlab # [ 0.499213] rcu: Hierarchical SRCU implementation. gitlab # [ 0.499600] rcu: Max phase no-delay instances is 400. gitlab # [ 0.500723] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.506167] smp: Bringing up secondary CPUs ... gitlab # [ 0.507013] smpboot: x86: Booting SMP configuration: gitlab # [ 0.507606] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.510740] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.512597] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) gitlab # [ 0.513949] Memory: 6011420K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved) gitlab # [ 0.515597] devtmpfs: initialized gitlab # [ 0.516828] x86/mm: Memory block size: 128MB gitlab # [ 0.520675] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.521671] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.522689] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.523867] PM: RTC time: 16:17:10, date: 2026-10-02 gitlab # [ 0.527457] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.529719] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.530661] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.532694] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.533610] audit: initializing netlink subsys (disabled) gitlab # [ 0.534659] audit: type=2000 audit(1790957830.338:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.534891] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.536598] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.537597] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.538597] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.539597] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.540612] cpuidle: using governor menu gitlab # [ 0.542835] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.543848] PCI: Using configuration type 1 for base access gitlab # [ 0.544599] PCI: Using configuration type 1 for extended access gitlab # [ 0.545766] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.547651] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.548599] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.549597] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.550597] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.555807] ACPI: Added _OSI(Module Device) gitlab # [ 0.557607] ACPI: Added _OSI(Processor Device) gitlab # [ 0.559606] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.566547] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.572870] ACPI: Interpreter enabled gitlab # [ 0.573627] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.574398] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.575305] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.576605] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.579047] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.592193] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.595614] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.599901] acpiphp: Slot [3] registered gitlab # [ 0.601726] acpiphp: Slot [4] registered gitlab # [ 0.603694] acpiphp: Slot [5] registered gitlab # [ 0.605681] acpiphp: Slot [6] registered gitlab # [ 0.607666] acpiphp: Slot [7] registered gitlab # [ 0.608700] acpiphp: Slot [8] registered gitlab # [ 0.610666] acpiphp: Slot [9] registered gitlab # [ 0.612650] acpiphp: Slot [10] registered gitlab # [ 0.613663] acpiphp: Slot [11] registered gitlab # [ 0.615683] acpiphp: Slot [12] registered gitlab # [ 0.616651] acpiphp: Slot [13] registered gitlab # [ 0.618640] acpiphp: Slot [14] registered gitlab # [ 0.619657] acpiphp: Slot [15] registered gitlab # [ 0.620641] acpiphp: Slot [16] registered gitlab # [ 0.622648] acpiphp: Slot [17] registered gitlab # [ 0.623640] acpiphp: Slot [18] registered gitlab # [ 0.625664] acpiphp: Slot [19] registered gitlab # [ 0.626640] acpiphp: Slot [20] registered gitlab # [ 0.628640] acpiphp: Slot [21] registered gitlab # [ 0.629639] acpiphp: Slot [22] registered gitlab # [ 0.630657] acpiphp: Slot [23] registered gitlab # [ 0.631541] acpiphp: Slot [24] registered gitlab # [ 0.632171] acpiphp: Slot [25] registered gitlab # [ 0.633616] acpiphp: Slot [26] registered gitlab # [ 0.634389] acpiphp: Slot [27] registered gitlab # [ 0.634618] acpiphp: Slot [28] registered gitlab # [ 0.635380] acpiphp: Slot [29] registered gitlab # [ 0.636634] acpiphp: Slot [30] registered gitlab # [ 0.637378] acpiphp: Slot [31] registered gitlab # [ 0.637635] PCI host bridge to bus 0000:00 gitlab # [ 0.638388] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.639523] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.640517] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.642597] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.643644] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.645612] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.647029] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.648295] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.652704] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.660696] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.662672] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.665604] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.667602] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.670603] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.673081] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.679800] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.683735] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.687137] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.689613] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.692027] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.709668] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.710696] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.713672] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.716422] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.722675] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.733643] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.735619] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.737658] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.765619] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.770255] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.774631] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.775723] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.778660] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.786844] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.798624] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.800620] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.802658] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.809734] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.819845] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.822618] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.824657] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.830969] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.840634] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.842646] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.844648] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.848212] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.853638] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.854558] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.855419] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.858861] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.863631] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.864526] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.865385] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.866585] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.869864] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.872636] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.873620] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.875833] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.880642] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.881565] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.882417] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.884867] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.889635] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.891605] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.892619] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.897386] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.898578] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.899556] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.900745] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.902693] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.905762] iommu: Default domain type: Translated gitlab # [ 0.906254] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.907851] ACPI: bus type USB registered gitlab # [ 0.908668] usbcore: registered new interface driver usbfs gitlab # [ 0.909620] usbcore: registered new interface driver hub gitlab # [ 0.910561] usbcore: registered new device driver usb gitlab # [ 0.912584] NetLabel: Initializing gitlab # [ 0.913051] NetLabel: domain hash size = 128 gitlab # [ 0.914598] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.915626] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.916606] PCI: Using ACPI for IRQ routing gitlab # [ 0.917809] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.918593] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.918593] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.920601] vgaarb: loaded gitlab # [ 0.921362] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.922597] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.927699] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.929280] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.930053] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.931588] pnp: PnP ACPI init gitlab # [ 0.932766] pnp: PnP ACPI: found 6 devices gitlab # [ 0.943327] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.947523] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.950499] NET: Registered PF_INET protocol family gitlab # [ 0.955908] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.988284] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.989882] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.991325] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.995079] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.996489] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.997813] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.999192] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.000682] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.002071] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 1.003131] NET: Registered PF_XDP protocol family gitlab # [ 1.004049] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.005126] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.006204] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.007368] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.008543] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.009950] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.010983] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.013465] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.016719] PCI: CLS 0 bytes, default 64 gitlab # [ 1.017796] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.018124] Trying to unpack rootfs image as initramfs... gitlab # [ 1.018238] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 1.022091] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 1.043455] Initialise system trusted keyrings gitlab # [ 1.044880] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.056863] Key type asymmetric registered gitlab # [ 1.057666] Asymmetric key parser 'x509' registered gitlab # [ 1.058591] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.060513] io scheduler mq-deadline registered gitlab # [ 1.061358] io scheduler kyber registered gitlab # [ 1.068379] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.069823] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.072478] Linux agpgart interface v0.103 gitlab # [ 1.073297] ACPI: bus type drm_connector registered gitlab # [ 1.076652] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.077810] usbserial: USB Serial support registered for generic gitlab # [ 1.078872] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.078872] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.081266] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.082737] drop_monitor: Initializing network drop monitor service gitlab # [ 1.083979] NET: Registered PF_INET6 protocol family gitlab # [ 1.086022] Segment Routing with IPv6 gitlab # [ 1.086774] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.089178] IPI shorthand broadcast: enabled gitlab # [ 1.093860] sched_clock: Marking stable (946020396, 147547912)->(1118495448, -24927140) gitlab # [ 1.095804] registered taskstats version 1 gitlab # [ 1.096952] Loading compiled-in X.509 certificates gitlab # [ 1.107572] Demotion targets for Node 0: null gitlab # [ 1.108718] Key type .fscrypt registered gitlab # [ 1.109424] Key type fscrypt-provisioning registered gitlab # [ 1.110446] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.111407] ima: Allocated hash algorithm: sha1 gitlab # [ 1.112240] ima: No architecture policies found gitlab # [ 1.113434] PM: Magic number: 14:590:286 gitlab # [ 1.115199] RAS: Correctable Errors collector initialized. gitlab # [ 1.119815] clk: Disabling unused clocks gitlab # [ 1.120587] PM: genpd: Disabling unused power domains gitlab # [ 1.282957] Freeing initrd memory: 29392K gitlab # [ 1.286968] Freeing unused decrypted memory: 2028K gitlab # [ 1.290183] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.291403] Write protecting the kernel read-only data: 32768k gitlab # [ 1.293731] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.295513] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.347154] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.348379] Run /init as init process gitlab # [ 1.361601] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.383952] fuse: init (API version 7.45) gitlab # [ 1.392867] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.400726] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.404232] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.448694] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.785632] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.820485] systemd[1]: Detected virtualization kvm. gitlab # [ 1.821340] systemd[1]: Detected architecture x86-64. gitlab # [ 1.822306] systemd[1]: Running in initrd. gitlab # [ 1.823607] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.824963] systemd[1]: Hostname set to . gitlab # [ 1.926054] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.973648] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.987640] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.989043] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.990678] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.991918] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.993104] systemd[1]: Reached target Path Units. gitlab # [ 1.994046] systemd[1]: Reached target Slice Units. gitlab # [ 1.994999] systemd[1]: Reached target Swaps. gitlab # [ 1.995878] systemd[1]: Reached target Timer Units. gitlab # [ 1.996979] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.998270] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.999507] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.000594] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.001645] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.002653] systemd[1]: Reached target Socket Units. gitlab # [ 2.004953] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.007217] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.009301] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.027708] systemd[1]: Starting Journal Service... gitlab # [ 2.030696] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.033456] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.041649] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.046974] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.048962] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.052997] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.054513] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.059283] systemd-journald[92]: Collecting audit messages is disabled. gitlab # [ 2.063712] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.088742] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.092348] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.096918] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.098163] netfs: FS-Cache loaded gitlab # [ 2.099601] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.106539] 9pnet: Installing 9P2000 support gitlab # [ 2.106735] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.109655] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.112284] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.966377] systemd-modules-load[94]: Using 4 probe threads gitlab # [ 2.114333] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.968634] systemd-modules-load[94]: Inserted module 'virtio_balloon' gitlab # [ 2.116118] systemd[1]: Started Journal Service. gitlab # [ 1.969950] systemd-modules-load[94]: Inserted module 'dm_mod' gitlab # [ 1.971264] systemd-modules-load[94]: Inserted module 'virtio_gpu' gitlab # [ 1.972935] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.977068] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.978114] systemd[1]: Reached target Local File Systems. gitlab # [ 1.978923] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.979872] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.981152] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.989822] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.001493] systemd-udevd[111]: Using default interface naming scheme 'v261'. gitlab # [ 2.012247] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.051950] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.053184] systemd[1]: Reached target System Initialization. gitlab # [ 2.054050] systemd[1]: Reached target Basic System. gitlab # [ 2.269222] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.273833] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.284686] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.291064] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.294520] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.295809] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.298359] virtio_blk virtio5: [vdb] 11843992 512-byte logical blocks (6.06 GB/5.65 GiB) gitlab # [ 2.303770] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.305061] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.306570] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.307745] usb usb1: Product: UHCI Host Controller gitlab # [ 2.308595] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.309342] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.313329] SCSI subsystem initialized gitlab # [ 2.318091] hub 1-0:1.0: USB hub found gitlab # [ 2.318821] hub 1-0:1.0: 2 ports detected gitlab # [ 2.325416] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.327451] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.328225] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.227796] (udev-worker)[118]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.231453] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.232328] (udev-worker)[118]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.384861] scsi host0: ata_piix gitlab # [ 2.390589] scsi host1: ata_piix gitlab # [ 2.391224] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.392687] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.247453] (udev-worker)[117]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.395797] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.252725] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.255748] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.256819] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.258248] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.259684] systemd-vconsole-setup[144]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.261423] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.281109] systemd-fsck[158]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.284328] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.541649] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.557753] ata2: found unknown device (class 0) gitlab # [ 2.559268] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.561482] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.618489] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.647889] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.709154] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.710440] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.711518] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.712204] usb 1-1: Manufacturer: QEMU gitlab # [ 2.712819] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.742783] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.756305] usbcore: registered new interface driver usbhid gitlab # [ 2.758442] usbhid: USB HID core driver gitlab # [ 2.768005] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.772408] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.899391] systemd[1]: Mounting /sysroot... gitlab # [ 3.208092] EXT4-fs (vda): mounted filesystem e3d4486a-d114-4147-ad52-a3c4f7807bf9 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.065673] systemd[1]: Mounted /sysroot. gitlab # [ 3.067182] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.070548] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.072575] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.074591] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.083683] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.087662] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.091205] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.096070] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.097166] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.109831] systemd-sysroot-fstab-check[179]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.111667] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.261159] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.115050] systemd[1]: Reload requested from client PID 179 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.263350] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.118930] systemd[1]: Reloading... gitlab # [ 3.184646] systemd[1]: Reloading finished in 67 ms. gitlab # [ 3.205589] systemd-sysroot-fstab-check[179]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.208437] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.210310] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.212094] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.213816] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.216329] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.218335] systemd-sysroot-fstab-check[179]: Requesting swap.target/start/replace... gitlab # [ 3.220874] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.224405] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.227607] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.229696] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.233173] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.235368] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.899755] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.951923] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.954196] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.956615] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.958615] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.993140] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.995810] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.016105] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.017913] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.020385] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.051960] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.054129] systemd[1]: Stopped target Basic System. gitlab # [ 4.055899] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.057728] systemd[1]: Stopped target Path Units. gitlab # [ 4.059404] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.061919] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.064514] systemd[1]: Stopped target Slice Units. gitlab # [ 4.067322] systemd[1]: Stopped target Socket Units. gitlab # [ 4.071814] systemd[1]: Stopped target System Initialization. gitlab # [ 4.076279] systemd[1]: Stopped target Swaps. gitlab # [ 4.077945] systemd[1]: Stopped target Timer Units. gitlab # [ 4.080680] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.083800] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.086494] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.089274] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.091323] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.095317] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.097512] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.099897] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.101536] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.103659] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.105329] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.107575] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.109848] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.112077] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.114134] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.115595] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.117382] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.119257] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.120698] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.122685] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.123937] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.124887] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.126176] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.127354] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.128482] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.129707] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.130853] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.131652] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.132635] systemd[1]: Closed udev Control Socket. gitlab # [ 4.133366] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.134168] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.135312] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.136196] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.137309] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.138297] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.139272] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.140284] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.141349] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.142149] systemd[1]: Reached target Switch Root. gitlab # [ 4.142802] systemd[1]: Starting NixOS Activation... gitlab # [ 4.249911] initrd-nixos-activation-start[230]: booting system configuration /nix/store/njaigjkyls05gc5nhj73sfrdwxdhmf08-nixos-system-gitlab-test gitlab # [ 4.288911] initrd-nixos-activation-start[230]: running activation script... gitlab # [ 4.573597] initrd-nixos-activation-start[253]: setting up /etc... gitlab # [ 4.692063] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.693674] systemd[1]: Finished NixOS Activation. gitlab # [ 4.696109] systemd[1]: Starting Switch Root... gitlab # [ 4.730190] systemd[1]: Switching root. gitlab # [ 4.931053] systemd-journald[92]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.029615] NET: Registered PF_VSOCK protocol family gitlab # [ 5.393196] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.400852] systemd[1]: Detected virtualization kvm. gitlab # [ 5.402706] systemd[1]: Detected architecture x86-64. gitlab # [ 5.404709] systemd[1]: Detected first boot. gitlab # [ 5.407299] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.553198] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.628209] systemd[1]: Applying preset policy. gitlab # [ 5.772585] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.897011] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.898407] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.900670] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.902821] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.904477] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.906022] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.907015] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.908302] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.909513] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.910272] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.911147] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.912174] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.913077] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.913994] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.914948] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.915980] systemd[1]: Reached target Path Units. gitlab # [ 5.916830] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.917739] systemd[1]: Reached target Slice Units. gitlab # [ 5.918491] systemd[1]: Reached target Swaps. gitlab # [ 5.920029] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.922147] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.923652] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.925300] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.926358] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.928800] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.930470] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.932470] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.933770] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.935030] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.936477] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.939460] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.940690] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.941643] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.945032] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.947110] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.949449] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.952625] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.960760] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.969609] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 5.980646] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 5.983302] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.987612] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.992144] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.029703] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.031811] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.040532] systemd[1]: Starting Journal Service... gitlab # [ 6.044120] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.050166] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.053626] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.054607] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.058688] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.063807] systemd-journald[323]: Collecting audit messages is enabled. gitlab # [ 6.065654] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.066818] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.067890] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.069090] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.070212] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.071566] EXT4-fs (vda): re-mounted e3d4486a-d114-4147-ad52-a3c4f7807bf9. gitlab # [ 6.072014] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.075032] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.078004] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.079293] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.933992] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 5.935319] s[ 6.082715] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # ystemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.084160] systemd[1]: Started Journal Service. gitlab # [ 5.937944] systemd[1]: syste[ 6.084474] loop: module loaded gitlab # md-journald.service: Deactivated successfully. gitlab # [ 5.940712] systemd-modules-load[324]: Using 3 probe threads gitlab # [ 5.941930] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.944214] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.947159] systemd-modules-load[324]: Inserted module 'loop' gitlab # [ 5.948080] systemd-modules-load[324]: Inserted module 'tls' gitlab # [ 5.948953] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.949990] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.951624] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 5.953138] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.954279] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.958387] systemd[1]: Finished Load Kernel Modules. gitlab # [ 6.114394] systemd-journald[323]: Received client request to flush runtime journal. gitlab # [ 6.008261] systemd-oomd[326]: No swap; memory pressure usage will be degraded gitlab # [ 6.010328] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.011495] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.013027] systemd[1]: Starting Firewall... gitlab # [ 6.013853] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.014771] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.015664] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.016747] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.017672] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.018709] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.019738] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.020789] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.021811] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.022917] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.039375] systemd-udevd[354]: Using default interface naming scheme 'v261'. gitlab # [ 6.074455] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.088397] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.143562] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.167684] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.190885] (udev-worker)[426]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.193163] (udev-worker)[426]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.196169] (udev-worker)[412]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.204142] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.206401] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.209210] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.212419] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.214173] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.216993] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.238671] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.428006] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.436796] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.444364] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.445444] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.446422] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.446719] rtc_cmos 00:05: setting system clock to 2026-10-02T16:17:15 UTC (1790957835) gitlab # [ 6.450087] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.452454] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.457616] ACPI: button: Power Button [PWRF] gitlab # [ 6.460961] systemd-journald[323]: Time jumped backwards, rotating. gitlab # [ 6.466179] FDC 0 is a S82078B gitlab # [ 6.325531] systemd[1]: Finished Firewall. gitlab # [ 6.480526] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.484805] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.484875] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.493833] Console: switching to colour dummy device 80x25 gitlab # [ 6.496638] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.496873] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.496880] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.500847] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.501635] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.515506] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.517057] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.555143] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 6.556988] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.574398] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 6.434298] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.474407] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.476171] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.483324] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.635594] ppdev: user-space parallel port driver gitlab # [ 6.490451] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.492410] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.503635] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.665665] kvm_amd: TSC scaling supported gitlab # [ 6.666136] kvm_amd: Nested Virtualization enabled gitlab # [ 6.666751] kvm_amd: Nested Paging enabled gitlab # [ 6.667188] kvm_amd: LBR virtualization supported gitlab # [ 6.667781] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.668298] kvm_amd: Virtual GIF supported gitlab # [ 6.668838] kvm_amd: Virtual NMI enabled gitlab # [ 6.705308] EDAC MC: Ver: 3.0.0 gitlab # [ 6.585823] systemd-vconsole-setup[498]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.588348] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.754926] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.777379] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.779277] systemd[1]: Reached target Local File Systems. gitlab # [ 6.781372] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.783802] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.786178] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.788293] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.791480] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.797260] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.832285] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.833741] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.839898] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.844376] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.847032] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.867766] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.883033] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.884750] systemd[1]: Starting Update is Completed... gitlab # [ 6.906753] systemd[1]: Finished Update is Completed. gitlab # [ 6.934852] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.936207] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.241538] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.244327] systemd[1]: Reached target System Initialization. gitlab # [ 7.246814] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.249298] systemd[1]: Started logrotate.timer. gitlab # [ 7.250984] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.253351] systemd[1]: Reached target Timer Units. gitlab # [ 7.255285] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.257633] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.269833] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.271954] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.274889] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.279169] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.280944] systemd[1]: Reached target Socket Units. gitlab # [ 7.331271] dbus-broker-launch[580]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.334586] dbus-broker-launch[580]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.337400] dbus-broker-launch[580]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.342954] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.359231] dbus-broker-launch[580]: Ready gitlab # [ 7.362858] systemd[1]: Reached target Basic System. gitlab # [ 7.366067] systemd[1]: Started backdoor.service. gitlab # [ 7.369650] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.373430] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.384224] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.391484] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.395856] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.407252] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.410076] systemd[1]: Starting resolvconf update... gitlab # connecting to host... gitlab # [ 7.415715] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.418868] logrotate[585]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.419831] logrotate[585]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.421215] logrotate[585]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.422173] logrotate[585]: Reading state from file: /var/lib/logrotate.status gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.423775] logrotate[585]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.424916] logrotate[585]: Allocating hash table for state file, size 64 entries gitlab: connected to guest root shell gitlab # [ 7.425871] logrotate[585]: Handling 4 logs gitlab: (connecting took 8.08 seconds) gitlab # [ 7.427098] logrotate[585]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab: (finished: waiting for the VM to finish booting, in 8.08 seconds) gitlab # [ 7.427988] logrotate[585]: considering log /var/log/btmp gitlab # [ 7.429321] logrotate[585]: Creating new state gitlab # [ 7.430770] logrotate[585]: Now: 2026-10-02 16:17 gitlab # [ 7.431888] logrotate[585]: Last rotated at 2026-10-02 16:00 gitlab # [ 7.433040] logrotate[585]: log does not need rotating (log has already been rotated) gitlab # [ 7.433790] logrotate[585]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.434427] logrotate[585]: considering log /var/log/wtmp gitlab # [ 7.434858] logrotate[585]: Creating new state gitlab # [ 7.435506] logrotate[585]: Now: 2026-10-02 16:17 gitlab # [ 7.435915] logrotate[585]: Last rotated at 2026-10-02 16:00 gitlab # [ 7.436518] logrotate[585]: log does not need rotating (log has already been rotated) gitlab # [ 7.436985] logrotate[585]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.437594] logrotate[585]: switching euid from 0 to 165 and egid from 0 to 165 (pid 585) gitlab # [ 7.438080] logrotate[585]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.438500] logrotate[585]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.438968] logrotate[585]: Creating new state gitlab # [ 7.439564] logrotate[585]: switching euid from 165 to 0 and egid from 165 to 0 (pid 585) gitlab # [ 7.440061] logrotate[585]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.440484] logrotate[585]: switching euid from 0 to 60 and egid from 0 to 60 (pid 585) gitlab # [ 7.440799] logrotate[585]: considering log /var/log/nginx/*.log gitlab # [ 7.440892] logrotate[585]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.441190] logrotate[585]: Creating new state gitlab # [ 7.441642] logrotate[585]: not running postrotate script, since no logs were rotated gitlab # [ 7.442557] logrotate[585]: switching euid from 60 to 0 and egid from 60 to 0 (pid 585) gitlab # [ 7.460318] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.468482] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.480781] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.482104] nsncd[586]: Oct 02 16:17:16.681 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.484296] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.485244] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.487195] systemd[1]: Starting User Login Management... gitlab # [ 7.521558] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.522926] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.523731] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.524576] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.525502] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.527493] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.529343] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.530198] systemd[1]: nscd.service: Consumed 53ms CPU time over 138ms wall clock time, 7.3M memory peak, 5.5M read from disk. gitlab # [ 7.533940] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.535424] systemd-logind[632]: New seat seat0. gitlab # [ 7.540086] systemd-logind[632]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.541372] systemd-logind[632]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard) gitlab # [ 7.543257] systemd-logind[632]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.544897] systemd[1]: Started User Login Management. gitlab # [ 7.547808] systemd[1]: Starting linger-users.service... gitlab # [ 7.553574] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.554574] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.555240] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.555964] nsncd[677]: Oct 02 16:17:16.754 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.566540] systemd[1]: Finished resolvconf update. gitlab # [ 7.567338] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.567879] systemd[1]: Finished linger-users.service. gitlab # [ 7.569342] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.572235] systemd[1]: Starting DHCP Client... gitlab # [ 7.575021] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.578408] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.604630] network-addresses-eth1-start[703]: adding address 192.168.1.1/24... done gitlab # [ 7.607791] network-addresses-eth1-start[703]: adding address 2001:db8:1::1/64... done gitlab # [ 7.615975] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.644376] dhcpcd[710]: dhcpcd-10.3.2 starting gitlab # [ 7.646069] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.646632] systemd[1]: Reached target Network. gitlab # [ 7.649574] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.650772] dhcpcd[767]: dev: loaded udev gitlab # [ 7.651789] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.802114] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.802944] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.660508] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.661487] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.664289] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.673779] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.684096] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.688075] systemd[1]: Started Getty on tty1. gitlab # [ 7.688848] systemd[1]: Reached target Login Prompts. gitlab # [ 7.905153] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.909767] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.910779] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.912648] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.913860] cfg80211: failed to load regulatory.db gitlab # [ 7.784687] nginx-pre-start[792]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.786617] nginx-pre-start[792]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.794142] systemd[1]: Started Nginx Web Server. gitlab # [ 7.944485] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.798852] dhcpcd[767]: eth0: waiting for carrier gitlab # [ 7.800905] dhcpcd[767]: eth0: carrier acquired gitlab # [ 7.806220] dhcpcd[767]: DUID 00:01:00:01:32:52:95:8d:52:54:00:12:34:56 gitlab # [ 7.807145] dhcpcd[767]: eth0: IAID 00:12:34:56 gitlab # [ 7.807769] dhcpcd[767]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.818929] redis-server[795]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.821398] redis-server[795]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.822655] redis-server[795]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=795, just started gitlab # [ 7.824392] redis-server[795]: Configuration loaded gitlab # [ 7.825564] redis[795]: monotonic clock: POSIX clock_gettime gitlab # [ 7.826715] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.827775] redis[795]: Running mode=standalone, port=0. gitlab # [ 7.828831] redis[795]: Server initialized gitlab # [ 7.829623] redis[795]: Ready to accept connections unix gitlab # [ 7.830694] redis[795]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.840115] postgresql-pre-start[793]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.841926] postgresql-pre-start[793]: This user must also own the server process. gitlab # [ 7.846904] postgresql-pre-start[793]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.848468] postgresql-pre-start[793]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.849740] postgresql-pre-start[793]: The default text search configuration will be set to "english". gitlab # [ 7.851096] postgresql-pre-start[793]: Data page checksums are enabled. gitlab # [ 7.852173] postgresql-pre-start[793]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.853615] postgresql-pre-start[793]: creating subdirectories ... ok gitlab # [ 7.854589] postgresql-pre-start[793]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.881330] postgresql-pre-start[793]: selecting default "max_connections" ... 100 gitlab # [ 7.917141] postgresql-pre-start[793]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.213606] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.356947] postgresql-pre-start[793]: selecting default time zone ... UTC gitlab # [ 8.360592] postgresql-pre-start[793]: creating configuration files ... ok gitlab # [ 8.460457] gitlab-workhorse[828]: time="2026-10-02T16:17:17Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.461261] gitlab-workhorse[828]: time="2026-10-02T16:17:17Z" level=info msg="Using redis/go-redis" gitlab # [ 8.463450] gitlab-workhorse[828]: time="2026-10-02T16:17:17Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.539194] postgresql-pre-start[793]: running bootstrap script ... ok gitlab # [ 8.972551] postgresql-pre-start[793]: performing post-bootstrap initialization ... ok gitlab # [ 9.289697] postgresql-pre-start[793]: syncing data to disk ... ok gitlab # [ 9.291417] postgresql-pre-start[793]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.292114] postgresql-pre-start[793]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.293311] postgresql-pre-start[793]: Success. You can now start the database server using: gitlab # [ 9.294497] postgresql-pre-start[793]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.436347] postgres[855]: [855] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.440453] postgres[855]: [855] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.443149] postgres[855]: [855] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.445953] postgres[855]: [855] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.458136] postgres[864]: [864] LOG: database system was shut down at 2026-10-02 16:17:18 GMT gitlab # [ 9.467128] postgres[855]: [855] LOG: database system is ready to accept connections gitlab # [ 9.471133] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.476435] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.506565] dhcpcd[767]: eth0: soliciting an IPv6 router gitlab # [ 9.508201] dhcpcd[767]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.510185] dhcpcd[767]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.512823] dhcpcd[767]: eth0: adding route to fec0::/64 gitlab # [ 9.514394] dhcpcd[767]: eth0: adding default route via fe80::2 gitlab # [ 9.598300] dhcpcd[767]: eth0: soliciting a DHCP lease gitlab # [ 9.751944] NET: Registered PF_PACKET protocol family gitlab # [ 9.609532] dhcpcd[767]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.613750] dhcpcd[767]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.651283] postgresql-setup-start[888]: CREATE DATABASE gitlab # [ 9.671947] postgresql-setup-start[896]: CREATE ROLE gitlab # [ 9.682801] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[872]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.684333] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[872]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.685740] postgresql-setup-start[898]: ALTER DATABASE gitlab # [ 9.705405] postgresql-setup-start[908]: CREATE ROLE gitlab # [ 9.708294] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.709054] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.710567] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.728777] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.731115] systemd[1]: Started gitaly.service. gitlab # [ 9.732262] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.813442] gitlab-postgresql-start[927]: CREATE DATABASE gitlab # [ 9.845429] gitlab-postgresql-start[932]: CREATE EXTENSION gitlab # [ 9.926424] gitlab-postgresql-start[934]: CREATE EXTENSION gitlab # [ 9.930047] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.273639] gitaly[919]: time="2026-10-02T16:17:19.473Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=919 gitlab # [ 10.275160] gitaly[919]: time="2026-10-02T16:17:19.473Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=919 gitlab # [ 10.275873] gitaly[919]: time="2026-10-02T16:17:19.473Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=919 gitlab # [ 10.276650] gitaly[919]: time="2026-10-02T16:17:19.473Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=919 gitlab # [ 10.277503] gitaly[919]: time="2026-10-02T16:17:19.473Z" level=info msg="Starting Gitaly" pid=919 version= gitlab # [ 10.471814] gitaly[919]: time="2026-10-02T16:17:19.672Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=197 pid=919 gitlab # [ 10.473806] gitaly[919]: time="2026-10-02T16:17:19.674Z" level=info msg="finished initializing bootstrap" duration_ms=1 pid=919 gitlab # [ 10.476062] gitaly[919]: time="2026-10-02T16:17:19.676Z" level=info msg="finished initializing command factory" duration_ms=2 pid=919 gitlab # [ 10.476950] gitaly[919]: time="2026-10-02T16:17:19.676Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-919/git-exec-448638850.d/git pid=919 gitlab # [ 10.483612] gitaly[919]: time="2026-10-02T16:17:19.684Z" level=info msg="finished detecting git version" duration_ms=7 pid=919 gitlab # [ 10.484567] gitaly[919]: time="2026-10-02T16:17:19.684Z" level=info msg="using Git version" pid=919 version=2.53.g gitlab # [ 10.487342] gitaly[919]: time="2026-10-02T16:17:19.688Z" level=info msg="clearing disk cache object folder" pid=919 storage=default gitlab # [ 10.487970] gitaly[919]: time="2026-10-02T16:17:19.688Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1444735073 pid=919 storage=default gitlab # [ 10.488632] gitaly[919]: time="2026-10-02T16:17:19.688Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1444735073 pid=919 storage=default gitlab # [ 10.489219] gitaly[919]: time="2026-10-02T16:17:19.688Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=919 gitlab # [ 10.489670] gitaly[919]: time="2026-10-02T16:17:19.688Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=919 gitlab # [ 10.490414] gitaly[919]: time="2026-10-02T16:17:19.688Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1444735073 clear_duration_ms=0 pid=919 storage=default gitlab # [ 10.523728] gitaly[919]: time="2026-10-02T16:17:19.724Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=919 gitlab # [ 10.524581] gitaly[919]: time="2026-10-02T16:17:19.724Z" level=info msg="finished tempdir cleaner walk" pid=919 storage=default time_ms=0 gitlab # [ 10.525561] gitaly[919]: time="2026-10-02T16:17:19.724Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=919 schema=unix gitlab # [ 10.526209] gitaly[919]: time="2026-10-02T16:17:19.724Z" level=info msg="counting repositories" pid=919 gitlab # [ 10.526742] gitaly[919]: time="2026-10-02T16:17:19.724Z" level=info msg="listening at address" address=/run/gitaly/gitaly-919/sock.d/intern pid=919 schema=unix gitlab # [ 10.533577] gitaly[919]: time="2026-10-02T16:17:19.734Z" level=info msg="starting to count repositories" pid=919 storage_path=/var/gitlab/state/repositories gitlab # [ 10.534549] gitaly[919]: time="2026-10-02T16:17:19.734Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=919 storage_path=/var/gitlab/state/repositories gitlab # [ 10.535165] gitaly[919]: time="2026-10-02T16:17:19.734Z" level=info msg="completed counting all repositories" counting_duration=0 pid=919 gitlab # [ 10.535716] gitaly[919]: time="2026-10-02T16:17:19.735Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=919 gitlab # [ 10.536469] gitaly[919]: time="2026-10-02T16:17:19.735Z" level=info msg="Started Gitaly" duration_ms=261 pid=919 gitlab # [ 10.537105] gitaly[919]: time="2026-10-02T16:17:19.736Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=919 gitlab # [ 10.537622] gitaly[919]: time="2026-10-02T16:17:19.736Z" level=info msg="maintenance: daily scheduled" pid=919 scheduled="2026-10-03 12:00:00 +0000 UTC" gitlab # [ 11.216579] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.219124] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.250134] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.454757] gitaly[919]: time="2026-10-02T16:17:21.654Z" level=info msg="License database preloaded" duration_ms=2180 pid=919 gitlab # [ 14.629600] dhcpcd[767]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.631908] dhcpcd[767]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.633941] dhcpcd[767]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.750415] systemd[1]: Started DHCP Client. gitlab # [ 14.753050] systemd[1]: Reached target Network is Online. gitlab # [ 42.502674] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[926]: W, [2026-10-02T16:17:51.702454 #926] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 44.647766] postgres[1330]: [1330] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 44.648935] postgres[1330]: [1330] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 44.650281] postgres[1330]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 44.651150] postgres[1330]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 44.651926] postgres[1330]: attidentity AS identity, gitlab # [ 44.652445] postgres[1330]: attgenerated as attgenerated gitlab # [ 44.653483] postgres[1330]: FROM pg_attribute a gitlab # [ 44.654496] postgres[1330]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 44.655308] postgres[1330]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 44.656178] postgres[1330]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 44.656947] postgres[1330]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 44.657789] postgres[1330]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 44.658781] postgres[1330]: ORDER BY a.attnum gitlab # [ 44.659590] postgres[1330]: gitlab # [ 48.279505] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1363]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 48.281240] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1363]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.518729] gitlab-workhorse[828]: time="2026-10-02T16:17:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.527417] gitlab-workhorse[828]: time="2026-10-02T16:18:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 63.709653] postgres[1438]: [1438] ERROR: relation "table_without_models" does not exist gitlab # [ 63.712077] postgres[1438]: [1438] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.535311] gitlab-workhorse[828]: time="2026-10-02T16:18:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 75.816559] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 75.821712] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 75.824710] systemd[1]: Starting gitlab.service... gitlab # [ 76.192180] bundle[1614]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 76.193330] bundle[1614]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 76.194186] bundle[1614]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 78.545446] gitlab-workhorse[828]: time="2026-10-02T16:18:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 78.973870] sidekiq-cluster[1613]: {"severity":"INFO","time":"2026-10-02T16:18:28.157Z","message":"Starting cluster with 1 processes"} gitlab # [ 79.110182] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.291Z","pid":1614,"message":"* Enabling systemd notification integration"} gitlab # [ 79.112242] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"Puma starting in cluster mode..."} gitlab # [ 79.113505] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 79.114978] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 79.116509] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Min threads: 0"} gitlab # [ 79.117949] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Max threads: 4"} gitlab # [ 79.119508] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Environment: production"} gitlab # [ 79.120882] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Master PID: 1614"} gitlab # [ 79.122132] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Workers: 2"} gitlab # [ 79.122998] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 79.123966] bundle[1614]: {"timestamp":"2026-10-02T16:18:28.310Z","pid":1614,"message":"* Preloading application"} gitlab # [ 88.554894] gitlab-workhorse[828]: time="2026-10-02T16:18:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.566189] gitlab-workhorse[828]: time="2026-10-02T16:18:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 102.804658] bundle[1614]: W, [2026-10-02T16:18:52.004524 #1614] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 105.217769] sidekiq-cluster[1644]: 2026-10-02T16:18:54.417Z pid=1644 tid=wjo INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1644, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431632384} gitlab # [ 105.395360] sidekiq-cluster[1644]: W, [2026-10-02T16:18:54.595924 #1644] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 105.436110] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:54.636Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 107.509543] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.709Z","message":"GitLab reliable fetch activated!"} gitlab # [ 107.513747] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.709Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 107.532552] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.733Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 107.538290] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.735Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 107.540135] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.738Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 107.549752] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.750Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 107.553881] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.754Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 107.557907] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.758Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 107.561657] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.762Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 107.566121] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.766Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 107.570381] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.771Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 107.574537] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.775Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 107.581339] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.778Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 107.585811] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.781Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 107.586735] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.783Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 107.587904] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.785Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 107.598694] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.799Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 107.605324] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.806Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 107.612301] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.812Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 107.618633] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.819Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 107.625310] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.826Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 107.631556] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.832Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 107.638202] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.838Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 107.643736] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.844Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 107.649293] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.849Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 107.655325] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.855Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 107.660648] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.861Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 107.667367] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.867Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 107.673503] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.874Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 107.679206] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.879Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 107.684605] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.885Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 107.690449] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.891Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 107.695756] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.896Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 107.705926] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.906Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 107.711762] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.912Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 107.716367] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.917Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 107.721084] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.921Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 107.725763] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.926Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 107.731439] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.932Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 107.737988] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.938Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 107.742713] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.943Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 107.747059] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.947Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 107.751984] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.952Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 107.756178] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.956Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 107.760912] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.961Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 107.765091] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.965Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 107.770084] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.970Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 107.773398] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.974Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 107.776727] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.977Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 107.779822] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.980Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 107.783147] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.983Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 107.786882] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.987Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 107.789989] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.990Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 107.793266] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:56.993Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 107.802615] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.003Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 107.806231] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.006Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 107.809449] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.010Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 107.812688] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.013Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 107.815618] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.016Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 107.819138] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.019Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 107.821874] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.022Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 107.824866] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.025Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 107.827800] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.028Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 107.830549] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.031Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 107.835281] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.035Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 107.845385] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.046Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 107.848925] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.049Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 107.895335] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.094Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 107.900789] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.101Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 107.903291] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.104Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 107.911058] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.111Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 107.915954] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.116Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 107.918264] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.119Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 107.920722] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.121Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 107.924199] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.124Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 107.926239] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.127Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 107.929128] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.129Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 107.931335] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.132Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 107.934128] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.134Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 107.936230] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.136Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 107.938795] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.139Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 107.940800] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.141Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 107.942855] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.143Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 107.945730] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.146Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 107.949142] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.149Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 107.951170] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.151Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 107.953965] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.154Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 107.956027] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.156Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 107.958858] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.159Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 107.960935] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.161Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 107.963566] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.164Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 107.965616] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.166Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 107.968485] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.169Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 107.970501] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.171Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 107.972747] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.173Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 107.975370] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.176Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 107.977688] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.178Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 107.980729] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.181Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 107.982840] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.183Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 107.985517] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.186Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 107.995085] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.195Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 107.997106] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.197Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 107.999117] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.199Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 108.002235] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.202Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 108.005283] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.206Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 108.008127] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.208Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 108.010457] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.211Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 108.013047] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.213Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 108.015124] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.215Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 108.018205] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.218Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 108.020148] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:18:57.220Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 108.577202] gitlab-workhorse[828]: time="2026-10-02T16:18:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.588164] gitlab-workhorse[828]: time="2026-10-02T16:19:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.589429] gitlab-workhorse[828]: time="2026-10-02T16:19:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 130.564544] postgres[2100]: [2100] ERROR: relation "table_without_models" does not exist gitlab # [ 130.565479] postgres[2100]: [2100] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 131.109105] postgres[2120]: [2120] ERROR: relation "table_without_models" does not exist gitlab # [ 131.109926] postgres[2120]: [2120] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 132.988043] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:22.188Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 132.992689] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:22.193Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 132.995236] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:22.195Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 132.996976] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:22.195Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 132.999021] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:22.197Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 134.483992] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:23.683Z","message":"Cleaning working queues"} gitlab # [ 134.487633] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:23.685Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"96e0c752550bfd82970b3ee4","created_at":"2026-10-02T16:18:24.677Z","trace_propagation_headers":{"sentry-trace":"22a3159436344913bcd9563f24b541ed-d54107277be4468e","baggage":"sentry-trace_id=22a3159436344913bcd9563f24b541ed,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"9840a60aa853714f4c9b69c0cdb86630","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:18:24.687Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-96e0c752550bfd82970b3ee4: start","job_status":"start","queue_duration_s":58.998162,"scheduling_latency_s":58.998207} gitlab # [ 134.628161] bundle[1614]: {"timestamp":"2026-10-02T16:19:23.827Z","pid":1614,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 134.632434] bundle[1614]: {"timestamp":"2026-10-02T16:19:23.828Z","pid":1614,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 134.636431] bundle[1614]: {"timestamp":"2026-10-02T16:19:23.828Z","pid":1614,"message":"! #\u003cThread:0x00007e68c79d5248 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 134.647665] bundle[1614]: {"timestamp":"2026-10-02T16:19:23.828Z","pid":1614,"message":"! #\u003cThread:0x00007e68c7c020e8 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 134.658973] bundle[1614]: {"timestamp":"2026-10-02T16:19:23.828Z","pid":1614,"message":"Use Ctrl-C to stop"} gitlab # [ 134.661983] bundle[1614]: {"timestamp":"2026-10-02T16:19:23.828Z","pid":1614,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 134.665416] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:23.832Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"96e0c752550bfd82970b3ee4","created_at":"2026-10-02T16:18:24.677Z","trace_propagation_headers":{"sentry-trace":"22a3159436344913bcd9563f24b541ed-d54107277be4468e","baggage":"sentry-trace_id=22a3159436344913bcd9563f24b541ed,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"9840a60aa853714f4c9b69c0cdb86630","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:18:24.687Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-96e0c752550bfd82970b3ee4: done: 0.147004 sec","job_status":"done","queue_duration_s":58.998162,"scheduling_latency_s":58.998207,"redis_calls":28,"redis_duration_s":0.004537,"redis_read_bytes":408,"redis_write_bytes":3436,"redis_feature_flag_calls":17,"redis_feature_flag_duration_s":0.002481,"redis_feature_flag_read_bytes":398,"redis_feature_flag_write_bytes":2189,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000821,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":13,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.078097356,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.147004,"completed_at":"2026-10-02T16:19:23.832Z","load_balancing_strategy":"primary","db_duration_s":0.04382,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 135.411557] bundle[1614]: {"timestamp":"2026-10-02T16:19:24.611Z","pid":1614,"message":"- Worker 0 (PID: 2194) booted in 0.72s, phase: 0"} gitlab # [ 135.454548] bundle[1614]: {"timestamp":"2026-10-02T16:19:24.655Z","pid":1614,"message":"- Worker 1 (PID: 2198) booted in 0.74s, phase: 0"} gitlab # [ 135.460912] systemd[1]: Started gitlab.service. gitlab # [ 135.462756] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 135.465241] systemd[1]: Starting setup-gitlab.service... gitlab # [ 135.490762] setup-gitlab-start[2231]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 137.418988] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:19:26 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 1905 gitlab # [ 137.423976] setup-gitlab-start[2231]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 137.426473] setup-gitlab-start[2231]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 137.438554] runuser[2245]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 137.832201] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 137.837189] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 137.841953] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.416746] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.422154] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.426890] setup-gitlab-start[2246]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 144.883383] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"d05a3dc901622a5c40a1d3fb","created_at":"2026-10-02T16:19:34.047Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b4f6c5ee8bf09a7080f24f8d2fc402e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.079Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-d05a3dc901622a5c40a1d3fb: start","job_status":"start","queue_duration_s":0.003436,"scheduling_latency_s":0.0035} gitlab # [ 144.932049] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.132Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 144.943935] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"d05a3dc901622a5c40a1d3fb","created_at":"2026-10-02T16:19:34.047Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b4f6c5ee8bf09a7080f24f8d2fc402e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.079Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-d05a3dc901622a5c40a1d3fb: done: 0.061462 sec","job_status":"done","queue_duration_s":0.003436,"scheduling_latency_s":0.0035,"redis_calls":10,"redis_duration_s":0.009256,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001901,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000600356,"duration_s":0.061462,"completed_at":"2026-10-02T16:19:34.144Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.990348] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"34fa3666a966e76c36bb307a","created_at":"2026-10-02T16:19:34.093Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5e7a33de98b0cd15153d605d0698ee3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.103Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-34fa3666a966e76c36bb307a: start","job_status":"start","queue_duration_s":0.042795,"scheduling_latency_s":0.042829} gitlab # [ 145.168489] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"34fa3666a966e76c36bb307a","created_at":"2026-10-02T16:19:34.093Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5e7a33de98b0cd15153d605d0698ee3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.103Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-34fa3666a966e76c36bb307a: done: 0.221413 sec","job_status":"done","queue_duration_s":0.042795,"scheduling_latency_s":0.042829,"redis_calls":12,"redis_duration_s":0.013655,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003301,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000597283,"exclusive_lock_hold_duration_s":0.129812309,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.221413,"completed_at":"2026-10-02T16:19:34.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.068056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.173456] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b56b8e0edfd7c416eebea1c7","created_at":"2026-10-02T16:19:34.116Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a51927284c25cc1a0c62c889e2e6d3be","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:19:34.175Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b56b8e0edfd7c416eebea1c7: start","job_status":"start","queue_duration_s":0.197932,"scheduling_latency_s":0.197968} gitlab # [ 145.193853] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.393Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 145.235974] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b56b8e0edfd7c416eebea1c7","created_at":"2026-10-02T16:19:34.116Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a51927284c25cc1a0c62c889e2e6d3be","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:19:34.175Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b56b8e0edfd7c416eebea1c7: done: 0.029077 sec","job_status":"done","queue_duration_s":0.197932,"scheduling_latency_s":0.197968,"redis_calls":10,"redis_duration_s":0.003351,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000708,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00072,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000517385,"duration_s":0.029077,"completed_at":"2026-10-02T16:19:34.402Z","load_balancing_strategy":"primary","db_duration_s":0.002255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.237583] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ff1c392dd098539f7361ada0","created_at":"2026-10-02T16:19:34.183Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.196Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ff1c392dd098539f7361ada0: start","job_status":"start","queue_duration_s":0.208357,"scheduling_latency_s":0.208384} gitlab # [ 145.330992] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ff1c392dd098539f7361ada0","created_at":"2026-10-02T16:19:34.183Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.196Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ff1c392dd098539f7361ada0: done: 0.126658 sec","job_status":"done","queue_duration_s":0.208357,"scheduling_latency_s":0.208384,"redis_calls":71,"redis_duration_s":0.027261,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004978,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.002975,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.018165,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001143,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000502857,"duration_s":0.126658,"completed_at":"2026-10-02T16:19:34.531Z","load_balancing_strategy":"primary","db_duration_s":0.006854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.374349] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.574Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"fece980166ea49b4adb6b65e","created_at":"2026-10-02T16:19:34.201Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3525badcd7fbc45d7d5a03e7d13790cd","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.215Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-fece980166ea49b4adb6b65e: start","job_status":"start","queue_duration_s":0.359549,"scheduling_latency_s":0.359589} gitlab # [ 145.485349] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"fece980166ea49b4adb6b65e","created_at":"2026-10-02T16:19:34.201Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3525badcd7fbc45d7d5a03e7d13790cd","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.215Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-fece980166ea49b4adb6b65e: done: 0.110721 sec","job_status":"done","queue_duration_s":0.359549,"scheduling_latency_s":0.359589,"redis_calls":13,"redis_duration_s":0.019667999999999998,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001992,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007446,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.01023,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.026161424,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.110721,"completed_at":"2026-10-02T16:19:34.685Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016264,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 145.553598] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.752Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1b03740cbed6bab23c38cdf9","created_at":"2026-10-02T16:19:34.237Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61b0b6908e1b3524f959e5fde9c5ea2a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.250Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1b03740cbed6bab23c38cdf9: start","job_status":"start","queue_duration_s":0.502142,"scheduling_latency_s":0.502263} gitlab # [ 145.656963] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.857Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1b03740cbed6bab23c38cdf9","created_at":"2026-10-02T16:19:34.237Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61b0b6908e1b3524f959e5fde9c5ea2a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.250Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1b03740cbed6bab23c38cdf9: done: 0.104333 sec","job_status":"done","queue_duration_s":0.502142,"scheduling_latency_s":0.502263,"redis_calls":16,"redis_duration_s":0.03048,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.007418,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.019734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003328,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001114108,"duration_s":0.104333,"completed_at":"2026-10-02T16:19:34.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.664693] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2f239602aa78b5fcb66b208a","created_at":"2026-10-02T16:19:34.257Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"317dfb9d6fec3f5c5eeb49122bd16737","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.265Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2f239602aa78b5fcb66b208a: start","job_status":"start","queue_duration_s":0.599644,"scheduling_latency_s":0.599687} gitlab # [ 145.749806] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2f239602aa78b5fcb66b208a","created_at":"2026-10-02T16:19:34.257Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"317dfb9d6fec3f5c5eeb49122bd16737","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.265Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2f239602aa78b5fcb66b208a: done: 0.085007 sec","job_status":"done","queue_duration_s":0.599644,"scheduling_latency_s":0.599687,"redis_calls":12,"redis_duration_s":0.004467,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000692,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002113,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001662,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000894247,"exclusive_lock_hold_duration_s":0.015395253,"duration_s":0.085007,"completed_at":"2026-10-02T16:19:34.949Z","load_balancing_strategy":"primary","db_duration_s":0.032959,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.754077] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:34.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"45d59d93f96c66401e93cfc0","created_at":"2026-10-02T16:19:34.275Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8f0fa320ce8a85929cb9bd0eef1eba7","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.280Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-45d59d93f96c66401e93cfc0: start","job_status":"start","queue_duration_s":0.673872,"scheduling_latency_s":0.6739} gitlab # [ 145.808692] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"45d59d93f96c66401e93cfc0","created_at":"2026-10-02T16:19:34.275Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8f0fa320ce8a85929cb9bd0eef1eba7","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.280Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-45d59d93f96c66401e93cfc0: done: 0.055279 sec","job_status":"done","queue_duration_s":0.673872,"scheduling_latency_s":0.6739,"redis_calls":12,"redis_duration_s":0.024792,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.011545,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009639,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003608,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000636953,"exclusive_lock_hold_duration_s":0.003059328,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.055279,"completed_at":"2026-10-02T16:19:35.009Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.809977] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a9762bae7f663c953211d939","created_at":"2026-10-02T16:19:34.284Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.294Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a9762bae7f663c953211d939: start","job_status":"start","queue_duration_s":0.715927,"scheduling_latency_s":0.715943} gitlab # [ 145.833987] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a9762bae7f663c953211d939","created_at":"2026-10-02T16:19:34.284Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.294Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a9762bae7f663c953211d939: done: 0.024106 sec","job_status":"done","queue_duration_s":0.715927,"scheduling_latency_s":0.715943,"redis_calls":19,"redis_duration_s":0.00278,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000674,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000222,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001429,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000246958,"duration_s":0.024106,"completed_at":"2026-10-02T16:19:35.034Z","load_balancing_strategy":"primary","db_duration_s":0.001501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.859771] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4ca380c048738a20de106a4e","created_at":"2026-10-02T16:19:34.318Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0daa9615631af0abec707832d3e73b68","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:19:34.329Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4ca380c048738a20de106a4e: start","job_status":"start","queue_duration_s":0.705886,"scheduling_latency_s":0.705901} gitlab # [ 145.871943] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.044Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 145.875032] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4ca380c048738a20de106a4e","created_at":"2026-10-02T16:19:34.318Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0daa9615631af0abec707832d3e73b68","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:19:34.329Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4ca380c048738a20de106a4e: done: 0.014512 sec","job_status":"done","queue_duration_s":0.705886,"scheduling_latency_s":0.705901,"redis_calls":10,"redis_duration_s":0.0016060000000000002,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000293,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037826,"duration_s":0.014512,"completed_at":"2026-10-02T16:19:35.049Z","load_balancing_strategy":"primary","db_duration_s":0.001872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.935915] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.051Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e33a43322714ada9951ab6f3","created_at":"2026-10-02T16:19:34.347Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea4097d6c4a0de29ff4e0d51508bdd83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:19:34.364Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e33a43322714ada9951ab6f3: start","job_status":"start","queue_duration_s":0.686066,"scheduling_latency_s":0.68608} gitlab # [ 145.936647] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e33a43322714ada9951ab6f3","created_at":"2026-10-02T16:19:34.347Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea4097d6c4a0de29ff4e0d51508bdd83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:19:34.364Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e33a43322714ada9951ab6f3: done: 0.023273 sec","job_status":"done","queue_duration_s":0.686066,"scheduling_latency_s":0.68608,"redis_calls":10,"redis_duration_s":0.001419,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000253664,"duration_s":0.023273,"completed_at":"2026-10-02T16:19:35.074Z","load_balancing_strategy":"primary","db_duration_s":0.004516,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.994605] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c08556db8b7327f9d9be4df3","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c08556db8b7327f9d9be4df3: start","job_status":"start","queue_duration_s":0.564418,"scheduling_latency_s":0.564431} gitlab # [ 145.995558] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c08556db8b7327f9d9be4df3","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c08556db8b7327f9d9be4df3: done: 0.016314 sec","job_status":"done","queue_duration_s":0.564418,"scheduling_latency_s":0.564431,"redis_calls":16,"redis_duration_s":0.002235,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001299,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208686,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016314,"completed_at":"2026-10-02T16:19:35.091Z","load_balancing_strategy":"primary","db_duration_s":0.001043,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.046196] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71a5f7f9f0cdf84167298637","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71a5f7f9f0cdf84167298637: start","job_status":"start","queue_duration_s":0.581636,"scheduling_latency_s":0.581651} gitlab # [ 146.047039] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71a5f7f9f0cdf84167298637","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71a5f7f9f0cdf84167298637: done: 0.010616 sec","job_status":"done","queue_duration_s":0.581636,"scheduling_latency_s":0.581651,"redis_calls":13,"redis_duration_s":0.001744,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010616,"completed_at":"2026-10-02T16:19:35.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.091962] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7807c73f91ac67cf8e5d2294","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7807c73f91ac67cf8e5d2294: start","job_status":"start","queue_duration_s":0.593297,"scheduling_latency_s":0.593312} gitlab # [ 146.092751] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7807c73f91ac67cf8e5d2294","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7807c73f91ac67cf8e5d2294: done: 0.010501 sec","job_status":"done","queue_duration_s":0.593297,"scheduling_latency_s":0.593312,"redis_calls":13,"redis_duration_s":0.001918,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001359,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010501,"completed_at":"2026-10-02T16:19:35.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.132170] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d67fa7cff1c43973637e6e5a","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d67fa7cff1c43973637e6e5a: start","job_status":"start","queue_duration_s":0.604787,"scheduling_latency_s":0.604801} gitlab # [ 146.132917] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d67fa7cff1c43973637e6e5a","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d67fa7cff1c43973637e6e5a: done: 0.010061 sec","job_status":"done","queue_duration_s":0.604787,"scheduling_latency_s":0.604801,"redis_calls":13,"redis_duration_s":0.0016910000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010061,"completed_at":"2026-10-02T16:19:35.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.177034] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c30beb64888a675923ce1a9b","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c30beb64888a675923ce1a9b: start","job_status":"start","queue_duration_s":0.615965,"scheduling_latency_s":0.615979} gitlab # [ 146.177925] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c30beb64888a675923ce1a9b","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c30beb64888a675923ce1a9b: done: 0.010382 sec","job_status":"done","queue_duration_s":0.615965,"scheduling_latency_s":0.615979,"redis_calls":13,"redis_duration_s":0.00172,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010382,"completed_at":"2026-10-02T16:19:35.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.194697] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d61cecee98d96f597d005369","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d61cecee98d96f597d005369: start","job_status":"start","queue_duration_s":0.627418,"scheduling_latency_s":0.627434} gitlab # [ 146.206252] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d61cecee98d96f597d005369","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d61cecee98d96f597d005369: done: 0.010388 sec","job_status":"done","queue_duration_s":0.627418,"scheduling_latency_s":0.627434,"redis_calls":13,"redis_duration_s":0.001703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010388,"completed_at":"2026-10-02T16:19:35.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.256725] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1687eca0a4f492d5cd53baf0","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1687eca0a4f492d5cd53baf0: start","job_status":"start","queue_duration_s":0.638888,"scheduling_latency_s":0.638903} gitlab # [ 146.257498] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1687eca0a4f492d5cd53baf0","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1687eca0a4f492d5cd53baf0: done: 0.010468 sec","job_status":"done","queue_duration_s":0.638888,"scheduling_latency_s":0.638903,"redis_calls":13,"redis_duration_s":0.001986,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001416,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010468,"completed_at":"2026-10-02T16:19:35.160Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.312116] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acdd41bd5bf59b0079de2cc6","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acdd41bd5bf59b0079de2cc6: start","job_status":"start","queue_duration_s":0.650449,"scheduling_latency_s":0.650464} gitlab # [ 146.312896] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acdd41bd5bf59b0079de2cc6","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acdd41bd5bf59b0079de2cc6: done: 0.010143 sec","job_status":"done","queue_duration_s":0.650449,"scheduling_latency_s":0.650464,"redis_calls":13,"redis_duration_s":0.0016950000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010143,"completed_at":"2026-10-02T16:19:35.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.359547] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5aee518e4e5ce88706bff67","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5aee518e4e5ce88706bff67: start","job_status":"start","queue_duration_s":0.661688,"scheduling_latency_s":0.661704} gitlab # [ 146.360340] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5aee518e4e5ce88706bff67","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5aee518e4e5ce88706bff67: done: 0.010221 sec","job_status":"done","queue_duration_s":0.661688,"scheduling_latency_s":0.661704,"redis_calls":13,"redis_duration_s":0.0016309999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010221,"completed_at":"2026-10-02T16:19:35.182Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.407213] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01d9ffa56f9ec6b157d37bd3","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01d9ffa56f9ec6b157d37bd3: start","job_status":"start","queue_duration_s":0.672771,"scheduling_latency_s":0.672785} gitlab # [ 146.407973] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01d9ffa56f9ec6b157d37bd3","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01d9ffa56f9ec6b157d37bd3: done: 0.01011 sec","job_status":"done","queue_duration_s":0.672771,"scheduling_latency_s":0.672785,"redis_calls":13,"redis_duration_s":0.0016359999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01011,"completed_at":"2026-10-02T16:19:35.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.463155] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"936e671848bc8f31b738ffab","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-936e671848bc8f31b738ffab: start","job_status":"start","queue_duration_s":0.68441,"scheduling_latency_s":0.684454} gitlab # [ 146.463901] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"936e671848bc8f31b738ffab","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-936e671848bc8f31b738ffab: done: 0.014867 sec","job_status":"done","queue_duration_s":0.68441,"scheduling_latency_s":0.684454,"redis_calls":13,"redis_duration_s":0.0016870000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014867,"completed_at":"2026-10-02T16:19:35.210Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.503678] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f7bda7bbbb77abdec82e492","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f7bda7bbbb77abdec82e492: start","job_status":"start","queue_duration_s":0.70011,"scheduling_latency_s":0.700124} gitlab # [ 146.504582] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f7bda7bbbb77abdec82e492","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f7bda7bbbb77abdec82e492: done: 0.010323 sec","job_status":"done","queue_duration_s":0.70011,"scheduling_latency_s":0.700124,"redis_calls":13,"redis_duration_s":0.0016539999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010323,"completed_at":"2026-10-02T16:19:35.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.544792] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96ceb7ecddd2e479c2606d81","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96ceb7ecddd2e479c2606d81: start","job_status":"start","queue_duration_s":0.711292,"scheduling_latency_s":0.711305} gitlab # [ 146.545757] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96ceb7ecddd2e479c2606d81","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96ceb7ecddd2e479c2606d81: done: 0.010009 sec","job_status":"done","queue_duration_s":0.711292,"scheduling_latency_s":0.711305,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010009,"completed_at":"2026-10-02T16:19:35.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.585686] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c77e85257bcf985c6a250ef","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c77e85257bcf985c6a250ef: start","job_status":"start","queue_duration_s":0.722156,"scheduling_latency_s":0.722171} gitlab # [ 146.586659] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c77e85257bcf985c6a250ef","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c77e85257bcf985c6a250ef: done: 0.009861 sec","job_status":"done","queue_duration_s":0.722156,"scheduling_latency_s":0.722171,"redis_calls":13,"redis_duration_s":0.001666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009861,"completed_at":"2026-10-02T16:19:35.242Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.611780] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03d12c56ef40c91c2a3d242d","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03d12c56ef40c91c2a3d242d: start","job_status":"start","queue_duration_s":0.733018,"scheduling_latency_s":0.733031} gitlab # [ 146.653232] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03d12c56ef40c91c2a3d242d","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03d12c56ef40c91c2a3d242d: done: 0.010176 sec","job_status":"done","queue_duration_s":0.733018,"scheduling_latency_s":0.733031,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010176,"completed_at":"2026-10-02T16:19:35.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.658174] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ad63dbbbe28e336e5c760d4","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ad63dbbbe28e336e5c760d4: start","job_status":"start","queue_duration_s":0.744332,"scheduling_latency_s":0.744346} gitlab # [ 146.669988] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ad63dbbbe28e336e5c760d4","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ad63dbbbe28e336e5c760d4: done: 0.009809 sec","job_status":"done","queue_duration_s":0.744332,"scheduling_latency_s":0.744346,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009809,"completed_at":"2026-10-02T16:19:35.265Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.712576] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9619539e34b482a350f0555","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9619539e34b482a350f0555: start","job_status":"start","queue_duration_s":0.755178,"scheduling_latency_s":0.755193} gitlab # [ 146.715482] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9619539e34b482a350f0555","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9619539e34b482a350f0555: done: 0.009751 sec","job_status":"done","queue_duration_s":0.755178,"scheduling_latency_s":0.755193,"redis_calls":13,"redis_duration_s":0.0016090000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009751,"completed_at":"2026-10-02T16:19:35.275Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.751394] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04a25095a89d3a9188e4bdd7","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04a25095a89d3a9188e4bdd7: start","job_status":"start","queue_duration_s":0.765933,"scheduling_latency_s":0.765948} gitlab # [ 146.762730] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04a25095a89d3a9188e4bdd7","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04a25095a89d3a9188e4bdd7: done: 0.009784 sec","job_status":"done","queue_duration_s":0.765933,"scheduling_latency_s":0.765948,"redis_calls":13,"redis_duration_s":0.001775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009784,"completed_at":"2026-10-02T16:19:35.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.818753] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f53974aedbe79f9719ced576","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f53974aedbe79f9719ced576: start","job_status":"start","queue_duration_s":0.776725,"scheduling_latency_s":0.776739} gitlab # [ 146.819627] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f53974aedbe79f9719ced576","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f53974aedbe79f9719ced576: done: 0.009999 sec","job_status":"done","queue_duration_s":0.776725,"scheduling_latency_s":0.776739,"redis_calls":13,"redis_duration_s":0.001666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009999,"completed_at":"2026-10-02T16:19:35.297Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.843701] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c856e79dbe51b965bf38de3f","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c856e79dbe51b965bf38de3f: start","job_status":"start","queue_duration_s":0.787557,"scheduling_latency_s":0.787571} gitlab # [ 146.891635] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c856e79dbe51b965bf38de3f","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c856e79dbe51b965bf38de3f: done: 0.010087 sec","job_status":"done","queue_duration_s":0.787557,"scheduling_latency_s":0.787571,"redis_calls":13,"redis_duration_s":0.0015739999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010087,"completed_at":"2026-10-02T16:19:35.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.892754] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab31f3aca54549b08a4669ea","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab31f3aca54549b08a4669ea: start","job_status":"start","queue_duration_s":0.798496,"scheduling_latency_s":0.798509} gitlab # [ 146.900944] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab31f3aca54549b08a4669ea","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab31f3aca54549b08a4669ea: done: 0.010127 sec","job_status":"done","queue_duration_s":0.798496,"scheduling_latency_s":0.798509,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010127,"completed_at":"2026-10-02T16:19:35.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.970947] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2de4a908ed8f728aecfa2220","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2de4a908ed8f728aecfa2220: start","job_status":"start","queue_duration_s":0.809459,"scheduling_latency_s":0.809473} gitlab # [ 146.971704] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2de4a908ed8f728aecfa2220","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2de4a908ed8f728aecfa2220: done: 0.01025 sec","job_status":"done","queue_duration_s":0.809459,"scheduling_latency_s":0.809473,"redis_calls":13,"redis_duration_s":0.001679,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01025,"completed_at":"2026-10-02T16:19:35.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.020731] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33ad3607f86899abf08b5cbd","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33ad3607f86899abf08b5cbd: start","job_status":"start","queue_duration_s":0.82055,"scheduling_latency_s":0.820563} gitlab # [ 147.021633] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33ad3607f86899abf08b5cbd","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33ad3607f86899abf08b5cbd: done: 0.009952 sec","job_status":"done","queue_duration_s":0.82055,"scheduling_latency_s":0.820563,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009952,"completed_at":"2026-10-02T16:19:35.341Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.034058] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdd8d22b2680659660fac0a4","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdd8d22b2680659660fac0a4: start","job_status":"start","queue_duration_s":0.831377,"scheduling_latency_s":0.831391} gitlab # [ 147.084831] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdd8d22b2680659660fac0a4","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdd8d22b2680659660fac0a4: done: 0.009889 sec","job_status":"done","queue_duration_s":0.831377,"scheduling_latency_s":0.831391,"redis_calls":13,"redis_duration_s":0.0015960000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009889,"completed_at":"2026-10-02T16:19:35.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.086057] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e59e7b4776e37a968707b79a","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e59e7b4776e37a968707b79a: start","job_status":"start","queue_duration_s":0.84213,"scheduling_latency_s":0.842144} gitlab # [ 147.095552] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e59e7b4776e37a968707b79a","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e59e7b4776e37a968707b79a: done: 0.010059 sec","job_status":"done","queue_duration_s":0.84213,"scheduling_latency_s":0.842144,"redis_calls":13,"redis_duration_s":0.0017369999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010059,"completed_at":"2026-10-02T16:19:35.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.148781] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84c04e5dec65d98def6fc4ef","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84c04e5dec65d98def6fc4ef: start","job_status":"start","queue_duration_s":0.853299,"scheduling_latency_s":0.853313} gitlab # [ 147.149682] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84c04e5dec65d98def6fc4ef","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84c04e5dec65d98def6fc4ef: done: 0.009816 sec","job_status":"done","queue_duration_s":0.853299,"scheduling_latency_s":0.853313,"redis_calls":13,"redis_duration_s":0.001616,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009816,"completed_at":"2026-10-02T16:19:35.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.215867] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"184c0ffa9210a1c34cc520f2","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-184c0ffa9210a1c34cc520f2: start","job_status":"start","queue_duration_s":0.864107,"scheduling_latency_s":0.864121} gitlab # [ 147.217115] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"184c0ffa9210a1c34cc520f2","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-184c0ffa9210a1c34cc520f2: done: 0.012279 sec","job_status":"done","queue_duration_s":0.864107,"scheduling_latency_s":0.864121,"redis_calls":13,"redis_duration_s":0.001613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012279,"completed_at":"2026-10-02T16:19:35.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.227809] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24bb3e81085f01d090ce2334","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24bb3e81085f01d090ce2334: start","job_status":"start","queue_duration_s":0.877419,"scheduling_latency_s":0.877434} gitlab # [ 147.239706] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24bb3e81085f01d090ce2334","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24bb3e81085f01d090ce2334: done: 0.009817 sec","job_status":"done","queue_duration_s":0.877419,"scheduling_latency_s":0.877434,"redis_calls":13,"redis_duration_s":0.0015530000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009817,"completed_at":"2026-10-02T16:19:35.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.279704] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a70ffa44ea001c90a32e1df","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a70ffa44ea001c90a32e1df: start","job_status":"start","queue_duration_s":0.88839,"scheduling_latency_s":0.888404} gitlab # [ 147.286097] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a70ffa44ea001c90a32e1df","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a70ffa44ea001c90a32e1df: done: 0.009821 sec","job_status":"done","queue_duration_s":0.88839,"scheduling_latency_s":0.888404,"redis_calls":13,"redis_duration_s":0.0016020000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009821,"completed_at":"2026-10-02T16:19:35.409Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.335049] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31a4671be7fff01c405a1d1b","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31a4671be7fff01c405a1d1b: start","job_status":"start","queue_duration_s":0.899218,"scheduling_latency_s":0.899231} gitlab # [ 147.335800] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31a4671be7fff01c405a1d1b","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31a4671be7fff01c405a1d1b: done: 0.009884 sec","job_status":"done","queue_duration_s":0.899218,"scheduling_latency_s":0.899231,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009884,"completed_at":"2026-10-02T16:19:35.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.389699] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad8843a35530f1e0a6adeb71","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad8843a35530f1e0a6adeb71: start","job_status":"start","queue_duration_s":0.910091,"scheduling_latency_s":0.910105} gitlab # [ 147.390574] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad8843a35530f1e0a6adeb71","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad8843a35530f1e0a6adeb71: done: 0.009799 sec","job_status":"done","queue_duration_s":0.910091,"scheduling_latency_s":0.910105,"redis_calls":13,"redis_duration_s":0.0016159999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009799,"completed_at":"2026-10-02T16:19:35.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.411234] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf111750a0b71432ef8682d7","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf111750a0b71432ef8682d7: start","job_status":"start","queue_duration_s":0.920886,"scheduling_latency_s":0.920902} gitlab # [ 147.422552] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf111750a0b71432ef8682d7","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf111750a0b71432ef8682d7: done: 0.009904 sec","job_status":"done","queue_duration_s":0.920886,"scheduling_latency_s":0.920902,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009904,"completed_at":"2026-10-02T16:19:35.441Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.462478] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ec813d845d5097bd331c6ca","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ec813d845d5097bd331c6ca: start","job_status":"start","queue_duration_s":0.931635,"scheduling_latency_s":0.931648} gitlab # [ 147.468633] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ec813d845d5097bd331c6ca","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ec813d845d5097bd331c6ca: done: 0.009858 sec","job_status":"done","queue_duration_s":0.931635,"scheduling_latency_s":0.931648,"redis_calls":13,"redis_duration_s":0.0015919999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009858,"completed_at":"2026-10-02T16:19:35.452Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.504218] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b411c527e4ca0cb61bcf1a5","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b411c527e4ca0cb61bcf1a5: start","job_status":"start","queue_duration_s":0.942339,"scheduling_latency_s":0.942353} gitlab # [ 147.515643] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b411c527e4ca0cb61bcf1a5","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b411c527e4ca0cb61bcf1a5: done: 0.010088 sec","job_status":"done","queue_duration_s":0.942339,"scheduling_latency_s":0.942353,"redis_calls":13,"redis_duration_s":0.001619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010088,"completed_at":"2026-10-02T16:19:35.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.551217] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ed757f6b30e1335dccc5f18","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ed757f6b30e1335dccc5f18: start","job_status":"start","queue_duration_s":0.953274,"scheduling_latency_s":0.953288} gitlab # [ 147.562169] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ed757f6b30e1335dccc5f18","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ed757f6b30e1335dccc5f18: done: 0.010269 sec","job_status":"done","queue_duration_s":0.953274,"scheduling_latency_s":0.953288,"redis_calls":13,"redis_duration_s":0.0015869999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010269,"completed_at":"2026-10-02T16:19:35.474Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.611172] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5e451eeadcf40336369c202","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5e451eeadcf40336369c202: start","job_status":"start","queue_duration_s":0.964395,"scheduling_latency_s":0.964408} gitlab # [ 147.611919] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5e451eeadcf40336369c202","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5e451eeadcf40336369c202: done: 0.009975 sec","job_status":"done","queue_duration_s":0.964395,"scheduling_latency_s":0.964408,"redis_calls":13,"redis_duration_s":0.001613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009975,"completed_at":"2026-10-02T16:19:35.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.659312] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59f7d7e5a4c5871bf854377b","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59f7d7e5a4c5871bf854377b: start","job_status":"start","queue_duration_s":0.975226,"scheduling_latency_s":0.975241} gitlab # [ 147.660058] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59f7d7e5a4c5871bf854377b","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59f7d7e5a4c5871bf854377b: done: 0.009914 sec","job_status":"done","queue_duration_s":0.975226,"scheduling_latency_s":0.975241,"redis_calls":13,"redis_duration_s":0.001656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009914,"completed_at":"2026-10-02T16:19:35.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.687938] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"380fda31c01267cb0ffcd9ca","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-380fda31c01267cb0ffcd9ca: start","job_status":"start","queue_duration_s":0.985985,"scheduling_latency_s":0.985998} gitlab # [ 147.699266] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"380fda31c01267cb0ffcd9ca","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-380fda31c01267cb0ffcd9ca: done: 0.009652 sec","job_status":"done","queue_duration_s":0.985985,"scheduling_latency_s":0.985998,"redis_calls":13,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009652,"completed_at":"2026-10-02T16:19:35.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.739817] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"478e1d710b499561e5e5b834","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-478e1d710b499561e5e5b834: start","job_status":"start","queue_duration_s":0.996768,"scheduling_latency_s":0.996782} gitlab # [ 147.744301] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"478e1d710b499561e5e5b834","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-478e1d710b499561e5e5b834: done: 0.00976 sec","job_status":"done","queue_duration_s":0.996768,"scheduling_latency_s":0.996782,"redis_calls":13,"redis_duration_s":0.001631,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00976,"completed_at":"2026-10-02T16:19:35.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.779912] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60767b55a151e8665b4d5aca","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60767b55a151e8665b4d5aca: start","job_status":"start","queue_duration_s":1.007509,"scheduling_latency_s":1.007523} gitlab # [ 147.821324] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60767b55a151e8665b4d5aca","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60767b55a151e8665b4d5aca: done: 0.009766 sec","job_status":"done","queue_duration_s":1.007509,"scheduling_latency_s":1.007523,"redis_calls":13,"redis_duration_s":0.00165,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009766,"completed_at":"2026-10-02T16:19:35.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.825797] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f57ad2c445db01cbbf9058c3","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f57ad2c445db01cbbf9058c3: start","job_status":"start","queue_duration_s":1.018247,"scheduling_latency_s":1.018262} gitlab # [ 147.864624] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f57ad2c445db01cbbf9058c3","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f57ad2c445db01cbbf9058c3: done: 0.009679 sec","job_status":"done","queue_duration_s":1.018247,"scheduling_latency_s":1.018262,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009679,"completed_at":"2026-10-02T16:19:35.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.871049] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c0e58f3c64d564c4e68f657","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c0e58f3c64d564c4e68f657: start","job_status":"start","queue_duration_s":1.02904,"scheduling_latency_s":1.029054} gitlab # [ 147.882288] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c0e58f3c64d564c4e68f657","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c0e58f3c64d564c4e68f657: done: 0.009663 sec","job_status":"done","queue_duration_s":1.02904,"scheduling_latency_s":1.029054,"redis_calls":13,"redis_duration_s":0.001585,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009663,"completed_at":"2026-10-02T16:19:35.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.924765] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"970818e52cdfad897788c56b","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-970818e52cdfad897788c56b: start","job_status":"start","queue_duration_s":1.039713,"scheduling_latency_s":1.039728} gitlab # [ 147.927885] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"970818e52cdfad897788c56b","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-970818e52cdfad897788c56b: done: 0.010077 sec","job_status":"done","queue_duration_s":1.039713,"scheduling_latency_s":1.039728,"redis_calls":13,"redis_duration_s":0.001569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010077,"completed_at":"2026-10-02T16:19:35.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.982422] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3d7939e7ab1db43b9532454","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3d7939e7ab1db43b9532454: start","job_status":"start","queue_duration_s":1.050626,"scheduling_latency_s":1.050638} gitlab # [ 147.983230] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3d7939e7ab1db43b9532454","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3d7939e7ab1db43b9532454: done: 0.010016 sec","job_status":"done","queue_duration_s":1.050626,"scheduling_latency_s":1.050638,"redis_calls":13,"redis_duration_s":0.0019030000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001386,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010016,"completed_at":"2026-10-02T16:19:35.571Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.008200] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d646109d671e9ea41688bf5c","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d646109d671e9ea41688bf5c: start","job_status":"start","queue_duration_s":1.061539,"scheduling_latency_s":1.061553} gitlab # [ 148.019433] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d646109d671e9ea41688bf5c","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d646109d671e9ea41688bf5c: done: 0.009969 sec","job_status":"done","queue_duration_s":1.061539,"scheduling_latency_s":1.061553,"redis_calls":13,"redis_duration_s":0.0017469999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009969,"completed_at":"2026-10-02T16:19:35.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.062742] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a275ff8e61a3bd079611243","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a275ff8e61a3bd079611243: start","job_status":"start","queue_duration_s":1.072346,"scheduling_latency_s":1.072359} gitlab # [ 148.064713] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a275ff8e61a3bd079611243","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a275ff8e61a3bd079611243: done: 0.009806 sec","job_status":"done","queue_duration_s":1.072346,"scheduling_latency_s":1.072359,"redis_calls":13,"redis_duration_s":0.001833,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009806,"completed_at":"2026-10-02T16:19:35.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.099855] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d783c75da924b60dfab2480f","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d783c75da924b60dfab2480f: start","job_status":"start","queue_duration_s":1.082998,"scheduling_latency_s":1.083012} gitlab # [ 148.143691] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d783c75da924b60dfab2480f","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d783c75da924b60dfab2480f: done: 0.009757 sec","job_status":"done","queue_duration_s":1.082998,"scheduling_latency_s":1.083012,"redis_calls":13,"redis_duration_s":0.0016150000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009757,"completed_at":"2026-10-02T16:19:35.603Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.145634] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48b07c0b56bcb0971db40ef3","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48b07c0b56bcb0971db40ef3: start","job_status":"start","queue_duration_s":1.093594,"scheduling_latency_s":1.093609} gitlab # [ 148.156708] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48b07c0b56bcb0971db40ef3","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48b07c0b56bcb0971db40ef3: done: 0.00994 sec","job_status":"done","queue_duration_s":1.093594,"scheduling_latency_s":1.093609,"redis_calls":13,"redis_duration_s":0.0016350000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00994,"completed_at":"2026-10-02T16:19:35.614Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.224385] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ff4cfdeb6982de8d4d1d9a0","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ff4cfdeb6982de8d4d1d9a0: start","job_status":"start","queue_duration_s":1.104523,"scheduling_latency_s":1.104537} gitlab # [ 148.225086] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ff4cfdeb6982de8d4d1d9a0","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ff4cfdeb6982de8d4d1d9a0: done: 0.009669 sec","job_status":"done","queue_duration_s":1.104523,"scheduling_latency_s":1.104537,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009669,"completed_at":"2026-10-02T16:19:35.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.265141] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae69bd3025c78d817d45ddd8","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae69bd3025c78d817d45ddd8: start","job_status":"start","queue_duration_s":1.115178,"scheduling_latency_s":1.115191} gitlab # [ 148.265773] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae69bd3025c78d817d45ddd8","created_at":"2026-10-02T16:19:34.433Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-494bf27e75554b4d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"235ea909d1fc920e65a2ad5d6043f16a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.511Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae69bd3025c78d817d45ddd8: done: 0.009747 sec","job_status":"done","queue_duration_s":1.115178,"scheduling_latency_s":1.115191,"redis_calls":13,"redis_duration_s":0.00169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009747,"completed_at":"2026-10-02T16:19:35.635Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.304572] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4e07df64f6541c1534af7684","created_at":"2026-10-02T16:19:34.501Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65ce25a5cf39db40f14a35b5c212ad19","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.513Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4e07df64f6541c1534af7684: start","job_status":"start","queue_duration_s":1.122957,"scheduling_latency_s":1.122971} gitlab # [ 148.305478] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.660Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4e07df64f6541c1534af7684","created_at":"2026-10-02T16:19:34.501Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65ce25a5cf39db40f14a35b5c212ad19","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.513Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4e07df64f6541c1534af7684: done: 0.023811 sec","job_status":"done","queue_duration_s":1.122957,"scheduling_latency_s":1.122971,"redis_calls":10,"redis_duration_s":0.001369,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217346,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.023811,"completed_at":"2026-10-02T16:19:35.660Z","load_balancing_strategy":"primary","db_duration_s":0.008098,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.358788] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"13e5b3881c921327870f3fe0","created_at":"2026-10-02T16:19:34.525Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"269f56cd17e6087fd5e5214e1842a718","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.572Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-13e5b3881c921327870f3fe0: start","job_status":"start","queue_duration_s":1.089005,"scheduling_latency_s":1.089027} gitlab # [ 148.359709] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"13e5b3881c921327870f3fe0","created_at":"2026-10-02T16:19:34.525Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"269f56cd17e6087fd5e5214e1842a718","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.572Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-13e5b3881c921327870f3fe0: done: 0.014813 sec","job_status":"done","queue_duration_s":1.089005,"scheduling_latency_s":1.089027,"redis_calls":12,"redis_duration_s":0.001724,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000529,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213715,"duration_s":0.014813,"completed_at":"2026-10-02T16:19:35.676Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.383518] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.677Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8a84eaee1a6865e12f887d2e","created_at":"2026-10-02T16:19:34.593Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd98e894ab79c318915bc86d7ab61592","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.609Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8a84eaee1a6865e12f887d2e: start","job_status":"start","queue_duration_s":1.068273,"scheduling_latency_s":1.068288} gitlab # [ 148.396869] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.684Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 148.400193] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.689Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8a84eaee1a6865e12f887d2e","created_at":"2026-10-02T16:19:34.593Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd98e894ab79c318915bc86d7ab61592","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.609Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8a84eaee1a6865e12f887d2e: done: 0.011849 sec","job_status":"done","queue_duration_s":1.068273,"scheduling_latency_s":1.068288,"redis_calls":10,"redis_duration_s":0.00129,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197511,"duration_s":0.011849,"completed_at":"2026-10-02T16:19:35.689Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.466938] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.690Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"56fe8e83d439836ca5739835","created_at":"2026-10-02T16:19:34.634Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce6854a3705398af4cfd88da7c175c7b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.645Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-56fe8e83d439836ca5739835: start","job_status":"start","queue_duration_s":1.044955,"scheduling_latency_s":1.044969} gitlab # [ 148.467677] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"56fe8e83d439836ca5739835","created_at":"2026-10-02T16:19:34.634Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce6854a3705398af4cfd88da7c175c7b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.645Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-56fe8e83d439836ca5739835: done: 0.011974 sec","job_status":"done","queue_duration_s":1.044955,"scheduling_latency_s":1.044969,"redis_calls":10,"redis_duration_s":0.001292,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197791,"duration_s":0.011974,"completed_at":"2026-10-02T16:19:35.702Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.489260] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2f6f7274796a4918a3274859","created_at":"2026-10-02T16:19:34.680Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"159b5b590b9a38e9b28a7e98adb183ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.760Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2f6f7274796a4918a3274859: start","job_status":"start","queue_duration_s":0.943467,"scheduling_latency_s":0.943481} gitlab # [ 148.501723] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.855Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2f6f7274796a4918a3274859","created_at":"2026-10-02T16:19:34.680Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"159b5b590b9a38e9b28a7e98adb183ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.760Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2f6f7274796a4918a3274859: done: 0.151738 sec","job_status":"done","queue_duration_s":0.943467,"scheduling_latency_s":0.943481,"redis_calls":14,"redis_duration_s":0.0021089999999999998,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000501,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000765,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000322667,"exclusive_lock_hold_duration_s":0.136197757,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.151738,"completed_at":"2026-10-02T16:19:35.855Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046311,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.548789] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"4c3faa6f77de842c9be66f0a","created_at":"2026-10-02T16:19:34.782Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb1b44eaf6907cad0206e4f5aff63e4b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.798Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-4c3faa6f77de842c9be66f0a: start","job_status":"start","queue_duration_s":1.059865,"scheduling_latency_s":1.059883} gitlab # [ 148.553791] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"4c3faa6f77de842c9be66f0a","created_at":"2026-10-02T16:19:34.782Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb1b44eaf6907cad0206e4f5aff63e4b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.798Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-4c3faa6f77de842c9be66f0a: done: 0.083996 sec","job_status":"done","queue_duration_s":1.059865,"scheduling_latency_s":1.059883,"redis_calls":20,"redis_duration_s":0.013452,"redis_read_bytes":20,"redis_write_bytes":3248,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003339,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.003919,"redis_queues_read_bytes":3,"redis_queues_write_bytes":818,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004498,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001696,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00081854,"duration_s":0.083996,"completed_at":"2026-10-02T16:19:35.942Z","load_balancing_strategy":"primary","db_duration_s":0.001941,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.629260] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.945Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"cfcbb955c338a78beb67c8a2","created_at":"2026-10-02T16:19:34.806Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf5f522d3c2ab708acebb17d55e0039b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.821Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-cfcbb955c338a78beb67c8a2: start","job_status":"start","queue_duration_s":1.12365,"scheduling_latency_s":1.123668} gitlab # [ 148.629969] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.958Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 148.630659] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"cfcbb955c338a78beb67c8a2","created_at":"2026-10-02T16:19:34.806Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf5f522d3c2ab708acebb17d55e0039b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.821Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-cfcbb955c338a78beb67c8a2: done: 0.020803 sec","job_status":"done","queue_duration_s":1.12365,"scheduling_latency_s":1.123668,"redis_calls":10,"redis_duration_s":0.006867,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001342,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0049,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00092833,"duration_s":0.020803,"completed_at":"2026-10-02T16:19:35.966Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.645518] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d9c31d5a74cbfb0427c3114d","created_at":"2026-10-02T16:19:34.842Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"902bf8eba28ac1ad65e598e962ed6c74","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.862Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d9c31d5a74cbfb0427c3114d: start","job_status":"start","queue_duration_s":1.107071,"scheduling_latency_s":1.107088} gitlab # [ 148.658667] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:35.996Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 148.661735] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d9c31d5a74cbfb0427c3114d","created_at":"2026-10-02T16:19:34.842Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"902bf8eba28ac1ad65e598e962ed6c74","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.862Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d9c31d5a74cbfb0427c3114d: done: 0.032459 sec","job_status":"done","queue_duration_s":1.107071,"scheduling_latency_s":1.107088,"redis_calls":10,"redis_duration_s":0.0038589999999999996,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000825,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002469,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000565,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000247797,"duration_s":0.032459,"completed_at":"2026-10-02T16:19:36.001Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.702098] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"afa9dd2fcca95620b0b48c54","created_at":"2026-10-02T16:19:34.909Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f7b0ebc3849da5cb79e662fcc08c542","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.926Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-afa9dd2fcca95620b0b48c54: start","job_status":"start","queue_duration_s":1.076634,"scheduling_latency_s":1.076647} gitlab # [ 148.712576] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.094Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"afa9dd2fcca95620b0b48c54","created_at":"2026-10-02T16:19:34.909Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f7b0ebc3849da5cb79e662fcc08c542","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.926Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-afa9dd2fcca95620b0b48c54: done: 0.091272 sec","job_status":"done","queue_duration_s":1.076634,"scheduling_latency_s":1.076647,"redis_calls":12,"redis_duration_s":0.002003,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000778,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000495594,"exclusive_lock_hold_duration_s":0.012035329,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.091272,"completed_at":"2026-10-02T16:19:36.093Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.759244] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3e2fcf1156617fa0c3c59c6b","created_at":"2026-10-02T16:19:34.944Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"774d6789a779b93c586b2b51c717ec33","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.964Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3e2fcf1156617fa0c3c59c6b: start","job_status":"start","queue_duration_s":1.133215,"scheduling_latency_s":1.133244} gitlab # [ 148.764666] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3e2fcf1156617fa0c3c59c6b","created_at":"2026-10-02T16:19:34.944Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"774d6789a779b93c586b2b51c717ec33","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.964Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3e2fcf1156617fa0c3c59c6b: done: 0.261212 sec","job_status":"done","queue_duration_s":1.133215,"scheduling_latency_s":1.133244,"redis_calls":565,"redis_duration_s":0.09269100000000001,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001067,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.088925,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.188666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000295569,"duration_s":0.261212,"completed_at":"2026-10-02T16:19:36.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.807052] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.359Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"11184d9b8b4c031c2572741b","created_at":"2026-10-02T16:19:34.983Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c57047da3160081528c5ee360d0bad1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.995Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-11184d9b8b4c031c2572741b: start","job_status":"start","queue_duration_s":1.364519,"scheduling_latency_s":1.364536} gitlab # [ 148.814241] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"11184d9b8b4c031c2572741b","created_at":"2026-10-02T16:19:34.983Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c57047da3160081528c5ee360d0bad1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:34.995Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-11184d9b8b4c031c2572741b: done: 0.018842 sec","job_status":"done","queue_duration_s":1.364519,"scheduling_latency_s":1.364536,"redis_calls":10,"redis_duration_s":0.001416,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000241372,"duration_s":0.018842,"completed_at":"2026-10-02T16:19:36.378Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003772,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.871677] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.379Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"59568738fc21730928ebb6c8","created_at":"2026-10-02T16:19:35.020Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-1a7703da8b974b1a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.029Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-59568738fc21730928ebb6c8: start","job_status":"start","queue_duration_s":1.350878,"scheduling_latency_s":1.350892} gitlab # [ 148.872564] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.391Z","message":"No existing merge request to be cleaned up."} gitlab # [ 148.873076] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.398Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"59568738fc21730928ebb6c8","created_at":"2026-10-02T16:19:35.020Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-1a7703da8b974b1a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.029Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-59568738fc21730928ebb6c8: done: 0.018924 sec","job_status":"done","queue_duration_s":1.350878,"scheduling_latency_s":1.350892,"redis_calls":13,"redis_duration_s":0.001931,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000253105,"duration_s":0.018924,"completed_at":"2026-10-02T16:19:36.398Z","load_balancing_strategy":"primary","db_duration_s":0.002395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.899670] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.399Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0bb5b8d4d58d447ea53f398f","created_at":"2026-10-02T16:19:35.020Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-1a7703da8b974b1a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.029Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0bb5b8d4d58d447ea53f398f: start","job_status":"start","queue_duration_s":1.370737,"scheduling_latency_s":1.37075} gitlab # [ 148.910672] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.406Z","message":"No existing merge request to be cleaned up."} gitlab # [ 148.912268] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.411Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0bb5b8d4d58d447ea53f398f","created_at":"2026-10-02T16:19:35.020Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-1a7703da8b974b1a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.029Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0bb5b8d4d58d447ea53f398f: done: 0.011808 sec","job_status":"done","queue_duration_s":1.370737,"scheduling_latency_s":1.37075,"redis_calls":10,"redis_duration_s":0.001511,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011808,"completed_at":"2026-10-02T16:19:36.411Z","load_balancing_strategy":"primary","db_duration_s":0.001045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.952666] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.412Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fd2d4a1f22676191bb7f5148","created_at":"2026-10-02T16:19:35.020Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-1a7703da8b974b1a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.029Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fd2d4a1f22676191bb7f5148: start","job_status":"start","queue_duration_s":1.383438,"scheduling_latency_s":1.383453} gitlab # [ 148.957430] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.421Z","message":"No existing merge request to be cleaned up."} gitlab # [ 148.958679] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.429Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fd2d4a1f22676191bb7f5148","created_at":"2026-10-02T16:19:35.020Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-1a7703da8b974b1a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.029Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fd2d4a1f22676191bb7f5148: done: 0.017459 sec","job_status":"done","queue_duration_s":1.383438,"scheduling_latency_s":1.383453,"redis_calls":10,"redis_duration_s":0.0041329999999999995,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003045,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017459,"completed_at":"2026-10-02T16:19:36.429Z","load_balancing_strategy":"primary","db_duration_s":0.003759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.001044] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.433Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a59799403aca67eba46a5e7a","created_at":"2026-10-02T16:19:35.020Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-1a7703da8b974b1a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.029Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a59799403aca67eba46a5e7a: start","job_status":"start","queue_duration_s":1.404177,"scheduling_latency_s":1.404193} gitlab # [ 149.003767] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.442Z","message":"No existing merge request to be cleaned up."} gitlab # [ 149.005522] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.447Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a59799403aca67eba46a5e7a","created_at":"2026-10-02T16:19:35.020Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-1a7703da8b974b1a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c8085e9bb8831f054967f6ab724df165","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.029Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a59799403aca67eba46a5e7a: done: 0.014268 sec","job_status":"done","queue_duration_s":1.404177,"scheduling_latency_s":1.404193,"redis_calls":10,"redis_duration_s":0.003133,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00164,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001493,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014268,"completed_at":"2026-10-02T16:19:36.447Z","load_balancing_strategy":"primary","db_duration_s":0.001234,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.070788] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"b39a91f36b21018428c1cc5f","created_at":"2026-10-02T16:19:35.876Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-6cb8e0e649c547ca","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"fb1b44eaf6907cad0206e4f5aff63e4b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.930Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-b39a91f36b21018428c1cc5f: start","job_status":"start","queue_duration_s":0.51829,"scheduling_latency_s":0.518305} gitlab # [ 149.071707] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:19:36.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"b39a91f36b21018428c1cc5f","created_at":"2026-10-02T16:19:35.876Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-6cb8e0e649c547ca","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"fb1b44eaf6907cad0206e4f5aff63e4b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:19:35.930Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-b39a91f36b21018428c1cc5f: done: 0.020248 sec","job_status":"done","queue_duration_s":0.51829,"scheduling_latency_s":0.518305,"redis_calls":13,"redis_duration_s":0.00182,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214553,"duration_s":0.020248,"completed_at":"2026-10-02T16:19:36.468Z","load_balancing_strategy":"primary","db_duration_s":0.002726,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.273162] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2844007306 wd_nsec: 2844007190 gitlab # [ 163.844408] setup-gitlab-start[2246]: W, [2026-10-02T16:19:53.044632 #2246] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 172.908925] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"333bae55ba3c1eaa572c0e36","created_at":"2026-10-02T16:20:02.049Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8cc37f5a2080827c1ebe728f43751e2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.104Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-333bae55ba3c1eaa572c0e36: start","job_status":"start","queue_duration_s":0.002822,"scheduling_latency_s":0.00288} gitlab # [ 173.025362] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"333bae55ba3c1eaa572c0e36","created_at":"2026-10-02T16:20:02.049Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8cc37f5a2080827c1ebe728f43751e2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.104Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-333bae55ba3c1eaa572c0e36: done: 0.117959 sec","job_status":"done","queue_duration_s":0.002822,"scheduling_latency_s":0.00288,"redis_calls":12,"redis_duration_s":0.017174000000000002,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001704,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010737,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002208661,"duration_s":0.117959,"completed_at":"2026-10-02T16:20:02.225Z","load_balancing_strategy":"primary","db_duration_s":0.015672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.098353] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0e80197f77b4c98122386f97","created_at":"2026-10-02T16:20:02.133Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21a3423e20531d170e1a33e161ecff50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.138Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0e80197f77b4c98122386f97: start","job_status":"start","queue_duration_s":0.160159,"scheduling_latency_s":0.16022} gitlab # [ 173.130568] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.331Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 173.158687] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0e80197f77b4c98122386f97","created_at":"2026-10-02T16:20:02.133Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21a3423e20531d170e1a33e161ecff50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.138Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0e80197f77b4c98122386f97: done: 0.060299 sec","job_status":"done","queue_duration_s":0.160159,"scheduling_latency_s":0.16022,"redis_calls":7,"redis_duration_s":0.010043,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008071,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001972,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060299,"completed_at":"2026-10-02T16:20:02.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.206215] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e678999ee466e7236e211381","created_at":"2026-10-02T16:20:02.154Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a9717d6cc86cacc587a6531534717f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.158Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e678999ee466e7236e211381: start","job_status":"start","queue_duration_s":0.248056,"scheduling_latency_s":0.248108} gitlab # [ 173.329865] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e678999ee466e7236e211381","created_at":"2026-10-02T16:20:02.154Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a9717d6cc86cacc587a6531534717f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.158Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e678999ee466e7236e211381: done: 0.123429 sec","job_status":"done","queue_duration_s":0.248056,"scheduling_latency_s":0.248108,"redis_calls":9,"redis_duration_s":0.023018,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.017322,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005696,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00216508,"exclusive_lock_hold_duration_s":0.034845491,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.123429,"completed_at":"2026-10-02T16:20:02.529Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.338818] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5ba92ab7696a6f12dd6497eb","created_at":"2026-10-02T16:20:02.165Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a34e336bf5b4c313a2e96dd9b0b3104a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:20:02.168Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5ba92ab7696a6f12dd6497eb: start","job_status":"start","queue_duration_s":0.369897,"scheduling_latency_s":0.369937} gitlab # [ 173.361150] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.561Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 173.371875] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.572Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5ba92ab7696a6f12dd6497eb","created_at":"2026-10-02T16:20:02.165Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a34e336bf5b4c313a2e96dd9b0b3104a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:20:02.168Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5ba92ab7696a6f12dd6497eb: done: 0.033771 sec","job_status":"done","queue_duration_s":0.369897,"scheduling_latency_s":0.369937,"redis_calls":7,"redis_duration_s":0.002683,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002035,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000648,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033771,"completed_at":"2026-10-02T16:20:02.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.389331] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.577Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::PublishEventWorker","jid":"c6069c95975db38547c8a84e","created_at":"2026-10-02T16:20:02.173Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5f1970a033f6df8f73b12acdf2634f0","meta.root_caller_id":"Cronjob","meta.feature_category":"release_orchestration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9f5dbd6b3e0f8a09b95f6e4bc38c93c1137695f37704af6aead7c9933a73fb55","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.182Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::PublishEventWorker JID-c6069c95975db38547c8a84e: start","job_status":"start","queue_duration_s":0.394525,"scheduling_latency_s":0.394555} gitlab # [ 173.450964] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::PublishEventWorker","jid":"c6069c95975db38547c8a84e","created_at":"2026-10-02T16:20:02.173Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5f1970a033f6df8f73b12acdf2634f0","meta.root_caller_id":"Cronjob","meta.feature_category":"release_orchestration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9f5dbd6b3e0f8a09b95f6e4bc38c93c1137695f37704af6aead7c9933a73fb55","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.182Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::PublishEventWorker JID-c6069c95975db38547c8a84e: done: 0.07392 sec","job_status":"done","queue_duration_s":0.394525,"scheduling_latency_s":0.394555,"redis_calls":10,"redis_duration_s":0.0065190000000000005,"redis_read_bytes":9,"redis_write_bytes":1039,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002321,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001155,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000808483,"duration_s":0.07392,"completed_at":"2026-10-02T16:20:02.651Z","load_balancing_strategy":"primary","db_duration_s":0.036435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.454056] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a5eb07cf40eb781c163e8e98","created_at":"2026-10-02T16:20:02.190Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.196Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a5eb07cf40eb781c163e8e98: start","job_status":"start","queue_duration_s":0.456453,"scheduling_latency_s":0.45647} gitlab # [ 173.565080] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a5eb07cf40eb781c163e8e98","created_at":"2026-10-02T16:20:02.190Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.196Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a5eb07cf40eb781c163e8e98: done: 0.112955 sec","job_status":"done","queue_duration_s":0.456453,"scheduling_latency_s":0.45647,"redis_calls":60,"redis_duration_s":0.026788,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.002601,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.022525,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001662,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.112955,"completed_at":"2026-10-02T16:20:02.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.611412] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"44c1156a9baec2de6f278862","created_at":"2026-10-02T16:20:02.209Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"728a2d80728bf567f720ae13441df820","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.220Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-44c1156a9baec2de6f278862: start","job_status":"start","queue_duration_s":0.59069,"scheduling_latency_s":0.590747} gitlab # [ 173.681429] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"44c1156a9baec2de6f278862","created_at":"2026-10-02T16:20:02.209Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"728a2d80728bf567f720ae13441df820","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.220Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-44c1156a9baec2de6f278862: done: 0.070436 sec","job_status":"done","queue_duration_s":0.59069,"scheduling_latency_s":0.590747,"redis_calls":10,"redis_duration_s":0.036042,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003459,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.032583,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00779317,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.070436,"completed_at":"2026-10-02T16:20:02.881Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005008,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 173.703037] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:02.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::CleanupPackageRegistryWorker","jid":"f86a0d152f04d25d9d18c2ff","created_at":"2026-10-02T16:20:02.297Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fba8e6108bca8ec2f7e839c2e2635db3","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31163b2078644913f89feb2245c5fa62ac9118d29e04cf14ce0b5def57bba8ea","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.312Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::CleanupPackageRegistryWorker JID-f86a0d152f04d25d9d18c2ff: start","job_status":"start","queue_duration_s":0.57245,"scheduling_latency_s":0.572471} gitlab # [ 173.837103] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::CleanupPackageRegistryWorker","jid":"f86a0d152f04d25d9d18c2ff","created_at":"2026-10-02T16:20:02.297Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fba8e6108bca8ec2f7e839c2e2635db3","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31163b2078644913f89feb2245c5fa62ac9118d29e04cf14ce0b5def57bba8ea","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.312Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::CleanupPackageRegistryWorker JID-f86a0d152f04d25d9d18c2ff: done: 0.152558 sec","job_status":"done","queue_duration_s":0.57245,"scheduling_latency_s":0.572471,"redis_calls":10,"redis_duration_s":0.022846,"redis_read_bytes":9,"redis_write_bytes":1125,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003096,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0008,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":10,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.055,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.055,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006246883,"extra.packages_cleanup_package_registry_worker.pending_destruction_package_files_count":0,"extra.packages_cleanup_package_registry_worker.processing_package_files_count":0,"extra.packages_cleanup_package_registry_worker.error_package_files_count":0,"extra.packages_cleanup_package_registry_worker.pending_cleanup_policies_count":0,"duration_s":0.152558,"completed_at":"2026-10-02T16:20:03.037Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.074418,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.903956] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.042Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"22e7aa5cda8c6f58494fba5f","created_at":"2026-10-02T16:20:02.326Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcd446bd4002175a881dfeda10b4e362","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.341Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-22e7aa5cda8c6f58494fba5f: start","job_status":"start","queue_duration_s":0.700578,"scheduling_latency_s":0.70061} gitlab # [ 173.916736] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.077Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"22e7aa5cda8c6f58494fba5f","created_at":"2026-10-02T16:20:02.326Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcd446bd4002175a881dfeda10b4e362","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.341Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-22e7aa5cda8c6f58494fba5f: done: 0.035611 sec","job_status":"done","queue_duration_s":0.700578,"scheduling_latency_s":0.70061,"redis_calls":7,"redis_duration_s":0.008583,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002198,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006385,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035611,"completed_at":"2026-10-02T16:20:03.077Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.953230] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.078Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"70de2ed23c8e3cea9ae5cbf8","created_at":"2026-10-02T16:20:02.351Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88c2875ffc8b889f3e71813806d51a3c","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.403Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-70de2ed23c8e3cea9ae5cbf8: start","job_status":"start","queue_duration_s":0.675279,"scheduling_latency_s":0.675295} gitlab # [ 173.965278] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"70de2ed23c8e3cea9ae5cbf8","created_at":"2026-10-02T16:20:02.351Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88c2875ffc8b889f3e71813806d51a3c","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.403Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-70de2ed23c8e3cea9ae5cbf8: done: 0.050958 sec","job_status":"done","queue_duration_s":0.675279,"scheduling_latency_s":0.675295,"redis_calls":10,"redis_duration_s":0.001439,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000228241,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.005116013},"duration_s":0.050958,"completed_at":"2026-10-02T16:20:03.129Z","load_balancing_strategy":"primary","db_duration_s":0.026504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.986696] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fbb86c96cfae4d4f725bb251","created_at":"2026-10-02T16:20:02.437Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f1a7e6c72431950974beaba3cc71c39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.444Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fbb86c96cfae4d4f725bb251: start","job_status":"start","queue_duration_s":0.68659,"scheduling_latency_s":0.686605} gitlab # [ 173.987660] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.175Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fbb86c96cfae4d4f725bb251","created_at":"2026-10-02T16:20:02.437Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f1a7e6c72431950974beaba3cc71c39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.444Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fbb86c96cfae4d4f725bb251: done: 0.04423 sec","job_status":"done","queue_duration_s":0.68659,"scheduling_latency_s":0.686605,"redis_calls":9,"redis_duration_s":0.0014,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000412064,"exclusive_lock_hold_duration_s":0.002654248,"duration_s":0.04423,"completed_at":"2026-10-02T16:20:03.175Z","load_balancing_strategy":"primary","db_duration_s":0.000861,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.012973] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"b2bc145e0c2644d770381497","created_at":"2026-10-02T16:20:02.462Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1de977e1f228a8d7e57eb4e363ee1e05","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.474Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-b2bc145e0c2644d770381497: start","job_status":"start","queue_duration_s":0.701924,"scheduling_latency_s":0.701938} gitlab # [ 174.056945] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.200Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"b2bc145e0c2644d770381497","created_at":"2026-10-02T16:20:02.462Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1de977e1f228a8d7e57eb4e363ee1e05","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.474Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-b2bc145e0c2644d770381497: done: 0.024209 sec","job_status":"done","queue_duration_s":0.701924,"scheduling_latency_s":0.701938,"redis_calls":10,"redis_duration_s":0.001366,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000252825,"duration_s":0.024209,"completed_at":"2026-10-02T16:20:03.200Z","load_balancing_strategy":"primary","db_duration_s":0.000481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.062310] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.201Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"9578bd83555bbd96fa3a49af","created_at":"2026-10-02T16:20:02.482Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9650b92ff40d8ef96c143e2d5593b9e4","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.490Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-9578bd83555bbd96fa3a49af: start","job_status":"start","queue_duration_s":0.710942,"scheduling_latency_s":0.710957} gitlab # [ 174.108165] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"9578bd83555bbd96fa3a49af","created_at":"2026-10-02T16:20:02.482Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9650b92ff40d8ef96c143e2d5593b9e4","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.490Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-9578bd83555bbd96fa3a49af: done: 0.012688 sec","job_status":"done","queue_duration_s":0.710942,"scheduling_latency_s":0.710957,"redis_calls":10,"redis_duration_s":0.001294,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208966,"duration_s":0.012688,"completed_at":"2026-10-02T16:20:03.214Z","load_balancing_strategy":"primary","db_duration_s":0.00114,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.112290] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5751c22ef33470c10d2c4c41","created_at":"2026-10-02T16:20:02.504Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"663e23d805463204731950cb98c43fe8","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.509Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5751c22ef33470c10d2c4c41: start","job_status":"start","queue_duration_s":0.705593,"scheduling_latency_s":0.705607} gitlab # [ 174.152122] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.224Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5751c22ef33470c10d2c4c41","created_at":"2026-10-02T16:20:02.504Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"663e23d805463204731950cb98c43fe8","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.509Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5751c22ef33470c10d2c4c41: done: 0.009852 sec","job_status":"done","queue_duration_s":0.705593,"scheduling_latency_s":0.705607,"redis_calls":9,"redis_duration_s":0.0012980000000000001,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000633,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000318476,"exclusive_lock_hold_duration_s":0.001572546,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.009852,"completed_at":"2026-10-02T16:20:03.224Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.163169] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6a0913b1db351dec53f643e9","created_at":"2026-10-02T16:20:02.525Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.527Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6a0913b1db351dec53f643e9: start","job_status":"start","queue_duration_s":0.697772,"scheduling_latency_s":0.697786} gitlab # [ 174.176970] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.240Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6a0913b1db351dec53f643e9","created_at":"2026-10-02T16:20:02.525Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.527Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6a0913b1db351dec53f643e9: done: 0.014912 sec","job_status":"done","queue_duration_s":0.697772,"scheduling_latency_s":0.697786,"redis_calls":14,"redis_duration_s":0.001956,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000187,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00122,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014912,"completed_at":"2026-10-02T16:20:03.240Z","load_balancing_strategy":"primary","db_duration_s":0.000671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.215661] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"e5fff14099c551e0075034bc","created_at":"2026-10-02T16:20:02.537Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebfe8fc2e7785eae4a1382c0036f1555","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.645Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-e5fff14099c551e0075034bc: start","job_status":"start","queue_duration_s":0.596377,"scheduling_latency_s":0.59639} gitlab # [ 174.262987] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"e5fff14099c551e0075034bc","created_at":"2026-10-02T16:20:02.537Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebfe8fc2e7785eae4a1382c0036f1555","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.645Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-e5fff14099c551e0075034bc: done: 0.01337 sec","job_status":"done","queue_duration_s":0.596377,"scheduling_latency_s":0.59639,"redis_calls":10,"redis_duration_s":0.001892,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000288584,"duration_s":0.01337,"completed_at":"2026-10-02T16:20:03.254Z","load_balancing_strategy":"primary","db_duration_s":0.000272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.264899] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1c5da5dae3e1dda301878169","created_at":"2026-10-02T16:20:02.656Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a73e6cd06a89876b0dd20580f71e63a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:20:02.663Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1c5da5dae3e1dda301878169: start","job_status":"start","queue_duration_s":0.592714,"scheduling_latency_s":0.592727} gitlab # [ 174.277712] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.260Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 174.280887] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1c5da5dae3e1dda301878169","created_at":"2026-10-02T16:20:02.656Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a73e6cd06a89876b0dd20580f71e63a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:20:02.663Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1c5da5dae3e1dda301878169: done: 0.007786 sec","job_status":"done","queue_duration_s":0.592714,"scheduling_latency_s":0.592727,"redis_calls":7,"redis_duration_s":0.0009599999999999999,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007786,"completed_at":"2026-10-02T16:20:03.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.330427] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.264Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"f4ef6570d7dada1702df9203","created_at":"2026-10-02T16:20:02.674Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"696dfe423dee8760b80fe511e04ed708","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.705Z","job_size_bytes":3,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-f4ef6570d7dada1702df9203: start","job_status":"start","queue_duration_s":0.559405,"scheduling_latency_s":0.559418} gitlab # [ 174.331106] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.275Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"f4ef6570d7dada1702df9203","created_at":"2026-10-02T16:20:02.674Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"696dfe423dee8760b80fe511e04ed708","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.705Z","job_size_bytes":3,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-f4ef6570d7dada1702df9203: done: 0.010868 sec","job_status":"done","queue_duration_s":0.559405,"scheduling_latency_s":0.559418,"redis_calls":10,"redis_duration_s":0.001344,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000317,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000209524,"duration_s":0.010868,"completed_at":"2026-10-02T16:20:03.275Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000301,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.375619] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.276Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"158fec45eb98466a42dcf567","created_at":"2026-10-02T16:20:02.711Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"375f8641379c9d4f37ca3348c09c8648","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:20:02.713Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-158fec45eb98466a42dcf567: start","job_status":"start","queue_duration_s":0.562626,"scheduling_latency_s":0.562639} gitlab # [ 174.378667] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"158fec45eb98466a42dcf567","created_at":"2026-10-02T16:20:02.711Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"375f8641379c9d4f37ca3348c09c8648","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:20:02.713Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-158fec45eb98466a42dcf567: done: 0.012427 sec","job_status":"done","queue_duration_s":0.562626,"scheduling_latency_s":0.562639,"redis_calls":7,"redis_duration_s":0.0009170000000000001,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012427,"completed_at":"2026-10-02T16:20:03.288Z","load_balancing_strategy":"primary","db_duration_s":0.001069,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.443566] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6fdf02a5b0cd73feac3b3fc6","created_at":"2026-10-02T16:20:02.727Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9de15fcea668be5a147737236fa515f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.729Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6fdf02a5b0cd73feac3b3fc6: start","job_status":"start","queue_duration_s":0.560459,"scheduling_latency_s":0.560473} gitlab # [ 174.444409] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6fdf02a5b0cd73feac3b3fc6","created_at":"2026-10-02T16:20:02.727Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9de15fcea668be5a147737236fa515f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.729Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6fdf02a5b0cd73feac3b3fc6: done: 0.012233 sec","job_status":"done","queue_duration_s":0.560459,"scheduling_latency_s":0.560473,"redis_calls":7,"redis_duration_s":0.001022,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000305,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.012233,"completed_at":"2026-10-02T16:20:03.301Z","load_balancing_strategy":"primary","db_duration_s":0.003671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.467141] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"41a1ea0aa7863524d0ef2931","created_at":"2026-10-02T16:20:02.735Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fca7613197e1056bb556756b1b3ae5e","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.743Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-41a1ea0aa7863524d0ef2931: start","job_status":"start","queue_duration_s":0.559823,"scheduling_latency_s":0.559837} gitlab # [ 174.510989] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"41a1ea0aa7863524d0ef2931","created_at":"2026-10-02T16:20:02.735Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fca7613197e1056bb556756b1b3ae5e","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.743Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-41a1ea0aa7863524d0ef2931: done: 0.016469 sec","job_status":"done","queue_duration_s":0.559823,"scheduling_latency_s":0.559837,"redis_calls":11,"redis_duration_s":0.001595,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000316,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208406,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.016469,"completed_at":"2026-10-02T16:20:03.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.520490] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.320Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a616844ac7dd7c90f70287d7","created_at":"2026-10-02T16:20:02.747Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6218e2567952a7737a014eb957f6b64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.750Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a616844ac7dd7c90f70287d7: start","job_status":"start","queue_duration_s":0.569692,"scheduling_latency_s":0.569707} gitlab # [ 174.571973] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.328Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a616844ac7dd7c90f70287d7","created_at":"2026-10-02T16:20:02.747Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6218e2567952a7737a014eb957f6b64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.750Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a616844ac7dd7c90f70287d7: done: 0.007989 sec","job_status":"done","queue_duration_s":0.569692,"scheduling_latency_s":0.569707,"redis_calls":7,"redis_duration_s":0.000955,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007989,"completed_at":"2026-10-02T16:20:03.328Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.573656] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3241ca4fa6a05ed843082892","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3241ca4fa6a05ed843082892: start","job_status":"start","queue_duration_s":0.574416,"scheduling_latency_s":0.574431} gitlab # [ 174.618920] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3241ca4fa6a05ed843082892","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3241ca4fa6a05ed843082892: done: 0.009498 sec","job_status":"done","queue_duration_s":0.574416,"scheduling_latency_s":0.574431,"redis_calls":13,"redis_duration_s":0.001626,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009498,"completed_at":"2026-10-02T16:20:03.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.620621] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"876cd86587beef7e275274a2","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-876cd86587beef7e275274a2: start","job_status":"start","queue_duration_s":0.584755,"scheduling_latency_s":0.584769} gitlab # [ 174.629711] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"876cd86587beef7e275274a2","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-876cd86587beef7e275274a2: done: 0.009199 sec","job_status":"done","queue_duration_s":0.584755,"scheduling_latency_s":0.584769,"redis_calls":13,"redis_duration_s":0.00157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009199,"completed_at":"2026-10-02T16:20:03.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.671187] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58015193d35d3edcf1b673f0","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58015193d35d3edcf1b673f0: start","job_status":"start","queue_duration_s":0.594795,"scheduling_latency_s":0.594807} gitlab # [ 174.675847] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58015193d35d3edcf1b673f0","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58015193d35d3edcf1b673f0: done: 0.009265 sec","job_status":"done","queue_duration_s":0.594795,"scheduling_latency_s":0.594807,"redis_calls":13,"redis_duration_s":0.0015869999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009265,"completed_at":"2026-10-02T16:20:03.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.714629] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e40fad92c9c5e76d3c13159","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e40fad92c9c5e76d3c13159: start","job_status":"start","queue_duration_s":0.604883,"scheduling_latency_s":0.604897} gitlab # [ 174.759601] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e40fad92c9c5e76d3c13159","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e40fad92c9c5e76d3c13159: done: 0.009048 sec","job_status":"done","queue_duration_s":0.604883,"scheduling_latency_s":0.604897,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009048,"completed_at":"2026-10-02T16:20:03.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.760816] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f1bc4c30136a2a33253b0b8","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f1bc4c30136a2a33253b0b8: start","job_status":"start","queue_duration_s":0.614767,"scheduling_latency_s":0.61478} gitlab # [ 174.772715] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f1bc4c30136a2a33253b0b8","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f1bc4c30136a2a33253b0b8: done: 0.009155 sec","job_status":"done","queue_duration_s":0.614767,"scheduling_latency_s":0.61478,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009155,"completed_at":"2026-10-02T16:20:03.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.842889] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea8c72e160aea77eb9ecc9bf","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea8c72e160aea77eb9ecc9bf: start","job_status":"start","queue_duration_s":0.624771,"scheduling_latency_s":0.624785} gitlab # [ 174.843773] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea8c72e160aea77eb9ecc9bf","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea8c72e160aea77eb9ecc9bf: done: 0.009273 sec","job_status":"done","queue_duration_s":0.624771,"scheduling_latency_s":0.624785,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009273,"completed_at":"2026-10-02T16:20:03.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.854607] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f47cec30b117ca0b42b47900","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f47cec30b117ca0b42b47900: start","job_status":"start","queue_duration_s":0.634881,"scheduling_latency_s":0.634896} gitlab # [ 174.866214] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f47cec30b117ca0b42b47900","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f47cec30b117ca0b42b47900: done: 0.013453 sec","job_status":"done","queue_duration_s":0.634881,"scheduling_latency_s":0.634896,"redis_calls":13,"redis_duration_s":0.0016350000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013453,"completed_at":"2026-10-02T16:20:03.403Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.918513] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23d1407f114696be90d826ab","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23d1407f114696be90d826ab: start","job_status":"start","queue_duration_s":0.649208,"scheduling_latency_s":0.649222} gitlab # [ 174.919286] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23d1407f114696be90d826ab","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23d1407f114696be90d826ab: done: 0.009294 sec","job_status":"done","queue_duration_s":0.649208,"scheduling_latency_s":0.649222,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009294,"completed_at":"2026-10-02T16:20:03.413Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.948915] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"859f95ffada97f55d36c7f40","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-859f95ffada97f55d36c7f40: start","job_status":"start","queue_duration_s":0.659354,"scheduling_latency_s":0.659368} gitlab # [ 174.960751] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"859f95ffada97f55d36c7f40","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-859f95ffada97f55d36c7f40: done: 0.009896 sec","job_status":"done","queue_duration_s":0.659354,"scheduling_latency_s":0.659368,"redis_calls":13,"redis_duration_s":0.0016279999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009896,"completed_at":"2026-10-02T16:20:03.424Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.023472] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db2a24a36075048db59693fe","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db2a24a36075048db59693fe: start","job_status":"start","queue_duration_s":0.670104,"scheduling_latency_s":0.670117} gitlab # [ 175.024098] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db2a24a36075048db59693fe","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db2a24a36075048db59693fe: done: 0.009771 sec","job_status":"done","queue_duration_s":0.670104,"scheduling_latency_s":0.670117,"redis_calls":13,"redis_duration_s":0.0019259999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001316,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009771,"completed_at":"2026-10-02T16:20:03.434Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.042677] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72f3403263a1f0035c3359ee","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72f3403263a1f0035c3359ee: start","job_status":"start","queue_duration_s":0.680794,"scheduling_latency_s":0.680807} gitlab # [ 175.053983] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72f3403263a1f0035c3359ee","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72f3403263a1f0035c3359ee: done: 0.009642 sec","job_status":"done","queue_duration_s":0.680794,"scheduling_latency_s":0.680807,"redis_calls":13,"redis_duration_s":0.001942,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009642,"completed_at":"2026-10-02T16:20:03.445Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.100303] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e5484a57e41d9f950991ac5","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e5484a57e41d9f950991ac5: start","job_status":"start","queue_duration_s":0.691335,"scheduling_latency_s":0.69135} gitlab # [ 175.100965] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e5484a57e41d9f950991ac5","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e5484a57e41d9f950991ac5: done: 0.009884 sec","job_status":"done","queue_duration_s":0.691335,"scheduling_latency_s":0.69135,"redis_calls":13,"redis_duration_s":0.002271,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001626,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009884,"completed_at":"2026-10-02T16:20:03.456Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.135179] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09b457d5c412398090ed2a0c","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09b457d5c412398090ed2a0c: start","job_status":"start","queue_duration_s":0.702251,"scheduling_latency_s":0.702265} gitlab # [ 175.146703] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09b457d5c412398090ed2a0c","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09b457d5c412398090ed2a0c: done: 0.009751 sec","job_status":"done","queue_duration_s":0.702251,"scheduling_latency_s":0.702265,"redis_calls":13,"redis_duration_s":0.002065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001409,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009751,"completed_at":"2026-10-02T16:20:03.466Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.192797] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"930ccce140310ec6bbfb17ed","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-930ccce140310ec6bbfb17ed: start","job_status":"start","queue_duration_s":0.712862,"scheduling_latency_s":0.712876} gitlab # [ 175.194451] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"930ccce140310ec6bbfb17ed","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-930ccce140310ec6bbfb17ed: done: 0.009313 sec","job_status":"done","queue_duration_s":0.712862,"scheduling_latency_s":0.712876,"redis_calls":13,"redis_duration_s":0.0016480000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009313,"completed_at":"2026-10-02T16:20:03.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.236307] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d483cac4d44d96da98710d51","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d483cac4d44d96da98710d51: start","job_status":"start","queue_duration_s":0.72302,"scheduling_latency_s":0.723033} gitlab # [ 175.278715] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d483cac4d44d96da98710d51","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d483cac4d44d96da98710d51: done: 0.009325 sec","job_status":"done","queue_duration_s":0.72302,"scheduling_latency_s":0.723033,"redis_calls":13,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009325,"completed_at":"2026-10-02T16:20:03.487Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.279934] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e0a6f7a2d44556b028741ea","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e0a6f7a2d44556b028741ea: start","job_status":"start","queue_duration_s":0.733193,"scheduling_latency_s":0.733207} gitlab # [ 175.289174] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e0a6f7a2d44556b028741ea","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e0a6f7a2d44556b028741ea: done: 0.009324 sec","job_status":"done","queue_duration_s":0.733193,"scheduling_latency_s":0.733207,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009324,"completed_at":"2026-10-02T16:20:03.497Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.338765] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8689148f20a06bd21482cec","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8689148f20a06bd21482cec: start","job_status":"start","queue_duration_s":0.743387,"scheduling_latency_s":0.7434} gitlab # [ 175.340148] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8689148f20a06bd21482cec","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8689148f20a06bd21482cec: done: 0.00937 sec","job_status":"done","queue_duration_s":0.743387,"scheduling_latency_s":0.7434,"redis_calls":13,"redis_duration_s":0.001626,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00937,"completed_at":"2026-10-02T16:20:03.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.392454] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab6d0c46161032da16063b89","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab6d0c46161032da16063b89: start","job_status":"start","queue_duration_s":0.753625,"scheduling_latency_s":0.753638} gitlab # [ 175.393173] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab6d0c46161032da16063b89","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab6d0c46161032da16063b89: done: 0.009438 sec","job_status":"done","queue_duration_s":0.753625,"scheduling_latency_s":0.753638,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009438,"completed_at":"2026-10-02T16:20:03.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.442253] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7532a587459581c607f08571","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7532a587459581c607f08571: start","job_status":"start","queue_duration_s":0.763887,"scheduling_latency_s":0.763899} gitlab # [ 175.443060] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7532a587459581c607f08571","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7532a587459581c607f08571: done: 0.009226 sec","job_status":"done","queue_duration_s":0.763887,"scheduling_latency_s":0.763899,"redis_calls":13,"redis_duration_s":0.001555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009226,"completed_at":"2026-10-02T16:20:03.527Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.493733] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f658f1b63ac98068621b384","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f658f1b63ac98068621b384: start","job_status":"start","queue_duration_s":0.773935,"scheduling_latency_s":0.773949} gitlab # [ 175.494702] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f658f1b63ac98068621b384","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f658f1b63ac98068621b384: done: 0.009548 sec","job_status":"done","queue_duration_s":0.773935,"scheduling_latency_s":0.773949,"redis_calls":13,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009548,"completed_at":"2026-10-02T16:20:03.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.545815] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14ec5c563d1d4cb9934b2cf1","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14ec5c563d1d4cb9934b2cf1: start","job_status":"start","queue_duration_s":0.784375,"scheduling_latency_s":0.784389} gitlab # [ 175.546762] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14ec5c563d1d4cb9934b2cf1","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14ec5c563d1d4cb9934b2cf1: done: 0.009052 sec","job_status":"done","queue_duration_s":0.784375,"scheduling_latency_s":0.784389,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009052,"completed_at":"2026-10-02T16:20:03.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.587563] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5245d5e89c31fa5e906663a","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5245d5e89c31fa5e906663a: start","job_status":"start","queue_duration_s":0.79427,"scheduling_latency_s":0.794283} gitlab # [ 175.588491] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5245d5e89c31fa5e906663a","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5245d5e89c31fa5e906663a: done: 0.009263 sec","job_status":"done","queue_duration_s":0.79427,"scheduling_latency_s":0.794283,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009263,"completed_at":"2026-10-02T16:20:03.558Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.604162] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cff83d87892039fcb35d785","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cff83d87892039fcb35d785: start","job_status":"start","queue_duration_s":0.804389,"scheduling_latency_s":0.804402} gitlab # [ 175.643761] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cff83d87892039fcb35d785","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cff83d87892039fcb35d785: done: 0.009189 sec","job_status":"done","queue_duration_s":0.804389,"scheduling_latency_s":0.804402,"redis_calls":13,"redis_duration_s":0.001617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009189,"completed_at":"2026-10-02T16:20:03.568Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.686255] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dc0fc72a481c09b24a0a1a9","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dc0fc72a481c09b24a0a1a9: start","job_status":"start","queue_duration_s":0.81443,"scheduling_latency_s":0.814444} gitlab # [ 175.686984] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dc0fc72a481c09b24a0a1a9","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dc0fc72a481c09b24a0a1a9: done: 0.009259 sec","job_status":"done","queue_duration_s":0.81443,"scheduling_latency_s":0.814444,"redis_calls":13,"redis_duration_s":0.001651,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009259,"completed_at":"2026-10-02T16:20:03.578Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.728661] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"579d1fe8d7e56c6fcd5586d6","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-579d1fe8d7e56c6fcd5586d6: start","job_status":"start","queue_duration_s":0.824547,"scheduling_latency_s":0.824561} gitlab # [ 175.729564] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"579d1fe8d7e56c6fcd5586d6","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-579d1fe8d7e56c6fcd5586d6: done: 0.009316 sec","job_status":"done","queue_duration_s":0.824547,"scheduling_latency_s":0.824561,"redis_calls":13,"redis_duration_s":0.001576,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009316,"completed_at":"2026-10-02T16:20:03.588Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.768752] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"319799fd696976892cbb7ce9","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-319799fd696976892cbb7ce9: start","job_status":"start","queue_duration_s":0.834696,"scheduling_latency_s":0.83471} gitlab # [ 175.769538] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"319799fd696976892cbb7ce9","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-319799fd696976892cbb7ce9: done: 0.009169 sec","job_status":"done","queue_duration_s":0.834696,"scheduling_latency_s":0.83471,"redis_calls":13,"redis_duration_s":0.0015199999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009169,"completed_at":"2026-10-02T16:20:03.598Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.820816] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f8e7439f8f74cdb45323500","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f8e7439f8f74cdb45323500: start","job_status":"start","queue_duration_s":0.844741,"scheduling_latency_s":0.844754} gitlab # [ 175.821717] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f8e7439f8f74cdb45323500","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f8e7439f8f74cdb45323500: done: 0.009199 sec","job_status":"done","queue_duration_s":0.844741,"scheduling_latency_s":0.844754,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009199,"completed_at":"2026-10-02T16:20:03.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.872681] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa89aa245cca015031c7ca9d","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa89aa245cca015031c7ca9d: start","job_status":"start","queue_duration_s":0.854771,"scheduling_latency_s":0.854785} gitlab # [ 175.873612] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa89aa245cca015031c7ca9d","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa89aa245cca015031c7ca9d: done: 0.009139 sec","job_status":"done","queue_duration_s":0.854771,"scheduling_latency_s":0.854785,"redis_calls":13,"redis_duration_s":0.00157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009139,"completed_at":"2026-10-02T16:20:03.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.882719] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"206e3d51be8f9fecd23785c6","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-206e3d51be8f9fecd23785c6: start","job_status":"start","queue_duration_s":0.864741,"scheduling_latency_s":0.864754} gitlab # [ 175.893926] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"206e3d51be8f9fecd23785c6","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-206e3d51be8f9fecd23785c6: done: 0.009008 sec","job_status":"done","queue_duration_s":0.864741,"scheduling_latency_s":0.864754,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009008,"completed_at":"2026-10-02T16:20:03.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.933795] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4016cc1fa0ba88f2aaaf0717","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4016cc1fa0ba88f2aaaf0717: start","job_status":"start","queue_duration_s":0.874595,"scheduling_latency_s":0.874609} gitlab # [ 175.940381] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4016cc1fa0ba88f2aaaf0717","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4016cc1fa0ba88f2aaaf0717: done: 0.009449 sec","job_status":"done","queue_duration_s":0.874595,"scheduling_latency_s":0.874609,"redis_calls":13,"redis_duration_s":0.001777,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009449,"completed_at":"2026-10-02T16:20:03.638Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.984330] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83f36fd80555be9cf936f782","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83f36fd80555be9cf936f782: start","job_status":"start","queue_duration_s":0.88486,"scheduling_latency_s":0.884874} gitlab # [ 175.986655] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83f36fd80555be9cf936f782","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83f36fd80555be9cf936f782: done: 0.008973 sec","job_status":"done","queue_duration_s":0.88486,"scheduling_latency_s":0.884874,"redis_calls":13,"redis_duration_s":0.001531,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008973,"completed_at":"2026-10-02T16:20:03.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.028050] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1c38e2383ca9fbf40cfcfcf","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1c38e2383ca9fbf40cfcfcf: start","job_status":"start","queue_duration_s":0.894688,"scheduling_latency_s":0.894701} gitlab # [ 176.069728] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1c38e2383ca9fbf40cfcfcf","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1c38e2383ca9fbf40cfcfcf: done: 0.009155 sec","job_status":"done","queue_duration_s":0.894688,"scheduling_latency_s":0.894701,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009155,"completed_at":"2026-10-02T16:20:03.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.070685] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2915dd64900eec343c75eeb","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2915dd64900eec343c75eeb: start","job_status":"start","queue_duration_s":0.904692,"scheduling_latency_s":0.904706} gitlab # [ 176.078813] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2915dd64900eec343c75eeb","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2915dd64900eec343c75eeb: done: 0.009196 sec","job_status":"done","queue_duration_s":0.904692,"scheduling_latency_s":0.904706,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009196,"completed_at":"2026-10-02T16:20:03.668Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.141443] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c48e131101548186f3800ba","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c48e131101548186f3800ba: start","job_status":"start","queue_duration_s":0.914726,"scheduling_latency_s":0.91474} gitlab # [ 176.142122] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c48e131101548186f3800ba","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c48e131101548186f3800ba: done: 0.009052 sec","job_status":"done","queue_duration_s":0.914726,"scheduling_latency_s":0.91474,"redis_calls":13,"redis_duration_s":0.0015149999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009052,"completed_at":"2026-10-02T16:20:03.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.187685] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e42b957e28a00e2f09bc607","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e42b957e28a00e2f09bc607: start","job_status":"start","queue_duration_s":0.924626,"scheduling_latency_s":0.92464} gitlab # [ 176.188177] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e42b957e28a00e2f09bc607","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e42b957e28a00e2f09bc607: done: 0.009256 sec","job_status":"done","queue_duration_s":0.924626,"scheduling_latency_s":0.92464,"redis_calls":13,"redis_duration_s":0.0016220000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009256,"completed_at":"2026-10-02T16:20:03.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.208802] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0d2d493c7b3caa3bb868fbb","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0d2d493c7b3caa3bb868fbb: start","job_status":"start","queue_duration_s":0.934718,"scheduling_latency_s":0.934732} gitlab # [ 176.220617] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0d2d493c7b3caa3bb868fbb","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0d2d493c7b3caa3bb868fbb: done: 0.009084 sec","job_status":"done","queue_duration_s":0.934718,"scheduling_latency_s":0.934732,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009084,"completed_at":"2026-10-02T16:20:03.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.263094] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f99b12618612fc0448495a19","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f99b12618612fc0448495a19: start","job_status":"start","queue_duration_s":0.944641,"scheduling_latency_s":0.944656} gitlab # [ 176.267314] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f99b12618612fc0448495a19","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f99b12618612fc0448495a19: done: 0.011258 sec","job_status":"done","queue_duration_s":0.944641,"scheduling_latency_s":0.944656,"redis_calls":13,"redis_duration_s":0.0016150000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011258,"completed_at":"2026-10-02T16:20:03.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.330699] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5a5e52b60c3a77c0296e25a","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5a5e52b60c3a77c0296e25a: start","job_status":"start","queue_duration_s":0.957014,"scheduling_latency_s":0.957028} gitlab # [ 176.331458] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5a5e52b60c3a77c0296e25a","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5a5e52b60c3a77c0296e25a: done: 0.017266 sec","job_status":"done","queue_duration_s":0.957014,"scheduling_latency_s":0.957028,"redis_calls":13,"redis_duration_s":0.0035250000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017266,"completed_at":"2026-10-02T16:20:03.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.350094] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1cbff025ba25cfcb6b07c3e","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1cbff025ba25cfcb6b07c3e: start","job_status":"start","queue_duration_s":0.979447,"scheduling_latency_s":0.979464} gitlab # [ 176.395576] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1cbff025ba25cfcb6b07c3e","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1cbff025ba25cfcb6b07c3e: done: 0.00969 sec","job_status":"done","queue_duration_s":0.979447,"scheduling_latency_s":0.979464,"redis_calls":13,"redis_duration_s":0.00158,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00969,"completed_at":"2026-10-02T16:20:03.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.396610] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ae3915794513268516f8b89","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ae3915794513268516f8b89: start","job_status":"start","queue_duration_s":0.98997,"scheduling_latency_s":0.990007} gitlab # [ 176.407588] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ae3915794513268516f8b89","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ae3915794513268516f8b89: done: 0.009214 sec","job_status":"done","queue_duration_s":0.98997,"scheduling_latency_s":0.990007,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009214,"completed_at":"2026-10-02T16:20:03.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.462693] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8a12c35ce7264fbe4f937e1","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8a12c35ce7264fbe4f937e1: start","job_status":"start","queue_duration_s":1.00004,"scheduling_latency_s":1.000054} gitlab # [ 176.463524] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8a12c35ce7264fbe4f937e1","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8a12c35ce7264fbe4f937e1: done: 0.009193 sec","job_status":"done","queue_duration_s":1.00004,"scheduling_latency_s":1.000054,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009193,"completed_at":"2026-10-02T16:20:03.764Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.515256] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41fe5cbe3e05b25da052a37d","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41fe5cbe3e05b25da052a37d: start","job_status":"start","queue_duration_s":1.010092,"scheduling_latency_s":1.010106} gitlab # [ 176.515923] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41fe5cbe3e05b25da052a37d","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41fe5cbe3e05b25da052a37d: done: 0.009134 sec","job_status":"done","queue_duration_s":1.010092,"scheduling_latency_s":1.010106,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009134,"completed_at":"2026-10-02T16:20:03.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.556981] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69de303a23cdbd883427be50","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69de303a23cdbd883427be50: start","job_status":"start","queue_duration_s":1.020162,"scheduling_latency_s":1.020177} gitlab # [ 176.557552] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69de303a23cdbd883427be50","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69de303a23cdbd883427be50: done: 0.009201 sec","job_status":"done","queue_duration_s":1.020162,"scheduling_latency_s":1.020177,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009201,"completed_at":"2026-10-02T16:20:03.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.582207] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67b4ab81212b78ec074c2c63","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67b4ab81212b78ec074c2c63: start","job_status":"start","queue_duration_s":1.030213,"scheduling_latency_s":1.030228} gitlab # [ 176.593624] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67b4ab81212b78ec074c2c63","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67b4ab81212b78ec074c2c63: done: 0.010356 sec","job_status":"done","queue_duration_s":1.030213,"scheduling_latency_s":1.030228,"redis_calls":13,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010356,"completed_at":"2026-10-02T16:20:03.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.632811] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ead4343ac3810c25bac7f35","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ead4343ac3810c25bac7f35: start","job_status":"start","queue_duration_s":1.041428,"scheduling_latency_s":1.041442} gitlab # [ 176.639704] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ead4343ac3810c25bac7f35","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ead4343ac3810c25bac7f35: done: 0.00925 sec","job_status":"done","queue_duration_s":1.041428,"scheduling_latency_s":1.041442,"redis_calls":13,"redis_duration_s":0.001602,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00925,"completed_at":"2026-10-02T16:20:03.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.684936] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dcccf0061c2c973a416c7e0","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dcccf0061c2c973a416c7e0: start","job_status":"start","queue_duration_s":1.051537,"scheduling_latency_s":1.051551} gitlab # [ 176.685747] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dcccf0061c2c973a416c7e0","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dcccf0061c2c973a416c7e0: done: 0.009326 sec","job_status":"done","queue_duration_s":1.051537,"scheduling_latency_s":1.051551,"redis_calls":13,"redis_duration_s":0.001577,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009326,"completed_at":"2026-10-02T16:20:03.815Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.748714] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5de42490205d0b0358a4ed49","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5de42490205d0b0358a4ed49: start","job_status":"start","queue_duration_s":1.061702,"scheduling_latency_s":1.061716} gitlab # [ 176.749692] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5de42490205d0b0358a4ed49","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5de42490205d0b0358a4ed49: done: 0.009299 sec","job_status":"done","queue_duration_s":1.061702,"scheduling_latency_s":1.061716,"redis_calls":13,"redis_duration_s":0.00163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009299,"completed_at":"2026-10-02T16:20:03.825Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.798626] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2de7be4a3f3bf3799e10bc07","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2de7be4a3f3bf3799e10bc07: start","job_status":"start","queue_duration_s":1.071852,"scheduling_latency_s":1.071865} gitlab # [ 176.799600] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2de7be4a3f3bf3799e10bc07","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2de7be4a3f3bf3799e10bc07: done: 0.009419 sec","job_status":"done","queue_duration_s":1.071852,"scheduling_latency_s":1.071865,"redis_calls":13,"redis_duration_s":0.0017560000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000728,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009419,"completed_at":"2026-10-02T16:20:03.836Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.839386] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"372da39c84aba871d7794fc6","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-372da39c84aba871d7794fc6: start","job_status":"start","queue_duration_s":1.082109,"scheduling_latency_s":1.082123} gitlab # [ 176.840159] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"372da39c84aba871d7794fc6","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-372da39c84aba871d7794fc6: done: 0.009091 sec","job_status":"done","queue_duration_s":1.082109,"scheduling_latency_s":1.082123,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009091,"completed_at":"2026-10-02T16:20:03.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.881942] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5a5b93fe269e7cdac2c27df","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5a5b93fe269e7cdac2c27df: start","job_status":"start","queue_duration_s":1.092056,"scheduling_latency_s":1.092069} gitlab # [ 176.882593] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5a5b93fe269e7cdac2c27df","created_at":"2026-10-02T16:20:02.665Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-85968a5314b847ff","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8708e27baf27748f5e295570369264b0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.754Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5a5b93fe269e7cdac2c27df: done: 0.046316 sec","job_status":"done","queue_duration_s":1.092056,"scheduling_latency_s":1.092069,"redis_calls":13,"redis_duration_s":0.001638,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.046316,"completed_at":"2026-10-02T16:20:03.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.906900] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b05f344d35b87e6a4208b9ba","created_at":"2026-10-02T16:20:02.759Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acf66300d955a944c8a318231b6fd5ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.761Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b05f344d35b87e6a4208b9ba: start","job_status":"start","queue_duration_s":1.132358,"scheduling_latency_s":1.132373} gitlab # [ 176.920160] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.898Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 176.923607] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b05f344d35b87e6a4208b9ba","created_at":"2026-10-02T16:20:02.759Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acf66300d955a944c8a318231b6fd5ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.761Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b05f344d35b87e6a4208b9ba: done: 0.008214 sec","job_status":"done","queue_duration_s":1.132358,"scheduling_latency_s":1.132373,"redis_calls":7,"redis_duration_s":0.000896,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008214,"completed_at":"2026-10-02T16:20:03.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.962968] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"24cae42fdde84cb6f2d7fe06","created_at":"2026-10-02T16:20:02.806Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ab5fef4ab5bd5a24dcb755ab90343c6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.810Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-24cae42fdde84cb6f2d7fe06: start","job_status":"start","queue_duration_s":1.092751,"scheduling_latency_s":1.092765} gitlab # [ 176.972046] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"24cae42fdde84cb6f2d7fe06","created_at":"2026-10-02T16:20:02.806Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ab5fef4ab5bd5a24dcb755ab90343c6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.810Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-24cae42fdde84cb6f2d7fe06: done: 0.008157 sec","job_status":"done","queue_duration_s":1.092751,"scheduling_latency_s":1.092765,"redis_calls":7,"redis_duration_s":0.000941,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008157,"completed_at":"2026-10-02T16:20:03.911Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.013195] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.912Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"1ebfd43b23d29cf9832e1b40","created_at":"2026-10-02T16:20:02.851Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bca5d1f417e144c638c2c50eadf0db15","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.866Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-1ebfd43b23d29cf9832e1b40: start","job_status":"start","queue_duration_s":1.045579,"scheduling_latency_s":1.045592} gitlab # [ 177.020382] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"1ebfd43b23d29cf9832e1b40","created_at":"2026-10-02T16:20:02.851Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bca5d1f417e144c638c2c50eadf0db15","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.866Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-1ebfd43b23d29cf9832e1b40: done: 0.022725 sec","job_status":"done","queue_duration_s":1.045579,"scheduling_latency_s":1.045592,"redis_calls":20,"redis_duration_s":0.002474,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000748,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000356,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217346,"duration_s":0.022725,"completed_at":"2026-10-02T16:20:03.935Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.071686] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"9f59d4d5bffc61e97634a35b","created_at":"2026-10-02T16:20:02.874Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67877e5dad4b0c62883736040c9afa24","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.883Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-9f59d4d5bffc61e97634a35b: start","job_status":"start","queue_duration_s":1.052551,"scheduling_latency_s":1.052564} gitlab # [ 177.072581] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.955Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"9f59d4d5bffc61e97634a35b","created_at":"2026-10-02T16:20:02.874Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67877e5dad4b0c62883736040c9afa24","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.883Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-9f59d4d5bffc61e97634a35b: done: 0.019424 sec","job_status":"done","queue_duration_s":1.052551,"scheduling_latency_s":1.052564,"redis_calls":10,"redis_duration_s":0.001284,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000244444,"duration_s":0.019424,"completed_at":"2026-10-02T16:20:03.955Z","load_balancing_strategy":"primary","db_duration_s":0.004317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.113677] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:03.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"acdf393aa34762836edf0145","created_at":"2026-10-02T16:20:02.903Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb751c071d8c826725e26490c5c53e59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.906Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-acdf393aa34762836edf0145: start","job_status":"start","queue_duration_s":1.049935,"scheduling_latency_s":1.049949} gitlab # [ 177.118148] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"acdf393aa34762836edf0145","created_at":"2026-10-02T16:20:02.903Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb751c071d8c826725e26490c5c53e59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.906Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-acdf393aa34762836edf0145: done: 0.160718 sec","job_status":"done","queue_duration_s":1.049935,"scheduling_latency_s":1.049949,"redis_calls":11,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000737,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000299759,"exclusive_lock_hold_duration_s":0.148620565,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.160718,"completed_at":"2026-10-02T16:20:04.117Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.065021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.173140] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"def0ab7880df76678278b83f","created_at":"2026-10-02T16:20:02.911Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f12ed15c8397878a1783a5baf984dd3","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.941Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-def0ab7880df76678278b83f: start","job_status":"start","queue_duration_s":1.176514,"scheduling_latency_s":1.17653} gitlab # [ 177.173852] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.136Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"def0ab7880df76678278b83f","created_at":"2026-10-02T16:20:02.911Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f12ed15c8397878a1783a5baf984dd3","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.941Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-def0ab7880df76678278b83f: done: 0.018499 sec","job_status":"done","queue_duration_s":1.176514,"scheduling_latency_s":1.17653,"redis_calls":13,"redis_duration_s":0.00165,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000733,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00063,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239137,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.018499,"completed_at":"2026-10-02T16:20:04.136Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002351,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.211979] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c6e83024be4d2c862c0d8eaf","created_at":"2026-10-02T16:20:02.957Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db16dda6ac9ea03bfd9155b1449c1e1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.963Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c6e83024be4d2c862c0d8eaf: start","job_status":"start","queue_duration_s":1.173791,"scheduling_latency_s":1.173805} gitlab # [ 177.221892] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.142Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 177.224723] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c6e83024be4d2c862c0d8eaf","created_at":"2026-10-02T16:20:02.957Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db16dda6ac9ea03bfd9155b1449c1e1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.963Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c6e83024be4d2c862c0d8eaf: done: 0.00769 sec","job_status":"done","queue_duration_s":1.173791,"scheduling_latency_s":1.173805,"redis_calls":7,"redis_duration_s":0.0009390000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.00769,"completed_at":"2026-10-02T16:20:04.145Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.273333] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7d7793715cf0e4725d4493d0","created_at":"2026-10-02T16:20:02.976Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05735814e399c0b5c4ce30d77c95ebd5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.981Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7d7793715cf0e4725d4493d0: start","job_status":"start","queue_duration_s":1.165037,"scheduling_latency_s":1.16505} gitlab # [ 177.274485] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.151Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 177.277557] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.154Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7d7793715cf0e4725d4493d0","created_at":"2026-10-02T16:20:02.976Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05735814e399c0b5c4ce30d77c95ebd5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:02.981Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7d7793715cf0e4725d4493d0: done: 0.008254 sec","job_status":"done","queue_duration_s":1.165037,"scheduling_latency_s":1.16505,"redis_calls":7,"redis_duration_s":0.000969,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008254,"completed_at":"2026-10-02T16:20:04.154Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.341125] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryCheck::DispatchWorker","jid":"22004997bfb3b520db6a7e39","created_at":"2026-10-02T16:20:02.996Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5649ea4b414487c7fa75b50d7e0ae149","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:bce0277309dfde25386eb0f6b49415f8ec269d6e1b0147ad102f2a38c86bd6e2","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.011Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::DispatchWorker JID-22004997bfb3b520db6a7e39: start","job_status":"start","queue_duration_s":1.144137,"scheduling_latency_s":1.144151} gitlab # [ 177.341812] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryCheck::DispatchWorker","jid":"22004997bfb3b520db6a7e39","created_at":"2026-10-02T16:20:02.996Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5649ea4b414487c7fa75b50d7e0ae149","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:bce0277309dfde25386eb0f6b49415f8ec269d6e1b0147ad102f2a38c86bd6e2","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.011Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::DispatchWorker JID-22004997bfb3b520db6a7e39: done: 0.057663 sec","job_status":"done","queue_duration_s":1.144137,"scheduling_latency_s":1.144151,"gitaly_calls":1,"gitaly_duration_s":0.024715,"redis_calls":26,"redis_duration_s":0.010723,"redis_read_bytes":45,"redis_write_bytes":3469,"redis_cache_calls":5,"redis_cache_duration_s":0.007819,"redis_cache_read_bytes":23,"redis_cache_write_bytes":95,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001082,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1190,"redis_queues_calls":2,"redis_queues_duration_s":0.000198,"redis_queues_read_bytes":3,"redis_queues_write_bytes":917,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":758,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000589,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":509,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.041914823999999996,"duration_s":0.057663,"completed_at":"2026-10-02T16:20:04.213Z","load_balancing_strategy":"primary","db_duration_s":0.00121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.364941] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8cdfb6af8bbf1f22825199a2","created_at":"2026-10-02T16:20:03.026Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30fac482d1ad4330c2c5497ee5b6b8da","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.040Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8cdfb6af8bbf1f22825199a2: start","job_status":"start","queue_duration_s":1.173846,"scheduling_latency_s":1.173874} gitlab # [ 177.406988] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.234Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8cdfb6af8bbf1f22825199a2","created_at":"2026-10-02T16:20:03.026Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30fac482d1ad4330c2c5497ee5b6b8da","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.040Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8cdfb6af8bbf1f22825199a2: done: 0.019745 sec","job_status":"done","queue_duration_s":1.173846,"scheduling_latency_s":1.173874,"redis_calls":10,"redis_duration_s":0.002452,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000282,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000221816,"duration_s":0.019745,"completed_at":"2026-10-02T16:20:04.234Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.415283] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6fdc7ac6ed6fa8a25d762e2e","created_at":"2026-10-02T16:20:03.057Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"415b64e92fed43b6655f6b83b48e2ede","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.094Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6fdc7ac6ed6fa8a25d762e2e: start","job_status":"start","queue_duration_s":1.141161,"scheduling_latency_s":1.141176} gitlab # [ 177.458695] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6fdc7ac6ed6fa8a25d762e2e","created_at":"2026-10-02T16:20:03.057Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"415b64e92fed43b6655f6b83b48e2ede","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.094Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6fdc7ac6ed6fa8a25d762e2e: done: 0.016715 sec","job_status":"done","queue_duration_s":1.141161,"scheduling_latency_s":1.141176,"redis_calls":12,"redis_duration_s":0.00162,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000609,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000355074,"exclusive_lock_hold_duration_s":0.0033590850000000004,"duration_s":0.016715,"completed_at":"2026-10-02T16:20:04.251Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.497542] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.253Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"80ab807fc95f3be4d06ba245","created_at":"2026-10-02T16:20:03.118Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34111df2e6c7b4e3975e28fe6c641b82","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.126Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-80ab807fc95f3be4d06ba245: start","job_status":"start","queue_duration_s":1.126441,"scheduling_latency_s":1.126456} gitlab # [ 177.498307] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"80ab807fc95f3be4d06ba245","created_at":"2026-10-02T16:20:03.118Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34111df2e6c7b4e3975e28fe6c641b82","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.126Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-80ab807fc95f3be4d06ba245: done: 0.03538 sec","job_status":"done","queue_duration_s":1.126441,"scheduling_latency_s":1.126456,"redis_calls":9,"redis_duration_s":0.001326,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000337194,"exclusive_lock_hold_duration_s":0.026245235,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.03538,"completed_at":"2026-10-02T16:20:04.288Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021865,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.515675] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4be48d35ed82ba838b667e42","created_at":"2026-10-02T16:20:03.134Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4363f4991ecdf571d4469b110f5f1df4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.136Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4be48d35ed82ba838b667e42: start","job_status":"start","queue_duration_s":1.152943,"scheduling_latency_s":1.152957} gitlab # [ 177.528105] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4be48d35ed82ba838b667e42","created_at":"2026-10-02T16:20:03.134Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4363f4991ecdf571d4469b110f5f1df4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.136Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4be48d35ed82ba838b667e42: done: 0.229282 sec","job_status":"done","queue_duration_s":1.152943,"scheduling_latency_s":1.152957,"redis_calls":562,"redis_duration_s":0.07144500000000001,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.07072,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.177765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.229282,"completed_at":"2026-10-02T16:20:04.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.576220] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.520Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"54b2135536eb31f842ab9f2d","created_at":"2026-10-02T16:20:03.166Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60f730dc355a83af10806ba0389e974d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.172Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-54b2135536eb31f842ab9f2d: start","job_status":"start","queue_duration_s":1.348139,"scheduling_latency_s":1.348161} gitlab # [ 177.576885] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"54b2135536eb31f842ab9f2d","created_at":"2026-10-02T16:20:03.166Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-769d3b90dff64ee3","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60f730dc355a83af10806ba0389e974d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.172Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-54b2135536eb31f842ab9f2d: done: 0.01098 sec","job_status":"done","queue_duration_s":1.348139,"scheduling_latency_s":1.348161,"redis_calls":7,"redis_duration_s":0.0009260000000000001,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01098,"completed_at":"2026-10-02T16:20:04.531Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.630077] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.532Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b3f6b4fa2f13869a6d0e343a","created_at":"2026-10-02T16:20:03.232Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-e330ad29e0d04b6d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.237Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b3f6b4fa2f13869a6d0e343a: start","job_status":"start","queue_duration_s":1.294724,"scheduling_latency_s":1.294738} gitlab # [ 177.630742] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.540Z","message":"No existing merge request to be cleaned up."} gitlab # [ 177.631440] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.551Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b3f6b4fa2f13869a6d0e343a","created_at":"2026-10-02T16:20:03.232Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-e330ad29e0d04b6d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.237Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b3f6b4fa2f13869a6d0e343a: done: 0.019478 sec","job_status":"done","queue_duration_s":1.294724,"scheduling_latency_s":1.294738,"redis_calls":10,"redis_duration_s":0.006202,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004665,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001537,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019478,"completed_at":"2026-10-02T16:20:04.551Z","load_balancing_strategy":"primary","db_duration_s":0.00336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.683888] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.554Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a5d76fb6476c236f68a1d749","created_at":"2026-10-02T16:20:03.232Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-e330ad29e0d04b6d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.237Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a5d76fb6476c236f68a1d749: start","job_status":"start","queue_duration_s":1.316665,"scheduling_latency_s":1.316682} gitlab # [ 177.684625] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.560Z","message":"No existing merge request to be cleaned up."} gitlab # [ 177.685126] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.565Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a5d76fb6476c236f68a1d749","created_at":"2026-10-02T16:20:03.232Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-e330ad29e0d04b6d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.237Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a5d76fb6476c236f68a1d749: done: 0.011018 sec","job_status":"done","queue_duration_s":1.316665,"scheduling_latency_s":1.316682,"redis_calls":10,"redis_duration_s":0.001323,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000782,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011018,"completed_at":"2026-10-02T16:20:04.565Z","load_balancing_strategy":"primary","db_duration_s":0.001095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.727221] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.566Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7fcf7fe8a0d1a748512b427c","created_at":"2026-10-02T16:20:03.232Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-e330ad29e0d04b6d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.237Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7fcf7fe8a0d1a748512b427c: start","job_status":"start","queue_duration_s":1.328508,"scheduling_latency_s":1.328521} gitlab # [ 177.727843] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.572Z","message":"No existing merge request to be cleaned up."} gitlab # [ 177.728522] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.576Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7fcf7fe8a0d1a748512b427c","created_at":"2026-10-02T16:20:03.232Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-e330ad29e0d04b6d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.237Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7fcf7fe8a0d1a748512b427c: done: 0.010696 sec","job_status":"done","queue_duration_s":1.328508,"scheduling_latency_s":1.328521,"redis_calls":10,"redis_duration_s":0.001303,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000783,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010696,"completed_at":"2026-10-02T16:20:04.576Z","load_balancing_strategy":"primary","db_duration_s":0.000979,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.750174] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.577Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"35f61c84c50c929c6e595fc1","created_at":"2026-10-02T16:20:03.232Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-e330ad29e0d04b6d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.237Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-35f61c84c50c929c6e595fc1: start","job_status":"start","queue_duration_s":1.340167,"scheduling_latency_s":1.340181} gitlab # [ 177.761421] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.583Z","message":"No existing merge request to be cleaned up."} gitlab # [ 177.800145] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.588Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"35f61c84c50c929c6e595fc1","created_at":"2026-10-02T16:20:03.232Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-e330ad29e0d04b6d","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"2a86d246ea27a1433d273d715aacac60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.237Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-35f61c84c50c929c6e595fc1: done: 0.010726 sec","job_status":"done","queue_duration_s":1.340167,"scheduling_latency_s":1.340181,"redis_calls":10,"redis_duration_s":0.001291,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000794,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010726,"completed_at":"2026-10-02T16:20:04.588Z","load_balancing_strategy":"primary","db_duration_s":0.000984,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.801173] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.589Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"4bb7d841f9b4fa976c1abe9a","created_at":"2026-10-02T16:20:03.921Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-d8fcc500a579480a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"bca5d1f417e144c638c2c50eadf0db15","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.926Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-4bb7d841f9b4fa976c1abe9a: start","job_status":"start","queue_duration_s":0.66235,"scheduling_latency_s":0.662364} gitlab # [ 177.844889] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.607Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"4bb7d841f9b4fa976c1abe9a","created_at":"2026-10-02T16:20:03.921Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-d8fcc500a579480a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"bca5d1f417e144c638c2c50eadf0db15","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.926Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-4bb7d841f9b4fa976c1abe9a: done: 0.018184 sec","job_status":"done","queue_duration_s":0.66235,"scheduling_latency_s":0.662364,"redis_calls":10,"redis_duration_s":0.002049,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213435,"duration_s":0.018184,"completed_at":"2026-10-02T16:20:04.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.847731] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.608Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"b99b7af7104894aaa3a6f663","created_at":"2026-10-02T16:20:03.927Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-d8fcc500a579480a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"bca5d1f417e144c638c2c50eadf0db15","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.930Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-b99b7af7104894aaa3a6f663: start","job_status":"start","queue_duration_s":0.678313,"scheduling_latency_s":0.678326} gitlab # [ 177.861594] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.621Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"b99b7af7104894aaa3a6f663","created_at":"2026-10-02T16:20:03.927Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-d8fcc500a579480a","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"bca5d1f417e144c638c2c50eadf0db15","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:03.930Z","job_size_bytes":2,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-b99b7af7104894aaa3a6f663: done: 0.012785 sec","job_status":"done","queue_duration_s":0.678313,"scheduling_latency_s":0.678326,"redis_calls":10,"redis_duration_s":0.001304,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226006,"duration_s":0.012785,"completed_at":"2026-10-02T16:20:04.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.925749] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"repository_check","args":["default"],"class":"RepositoryCheck::BatchWorker","jid":"7174cbaeca28896b5c8c1c17","created_at":"2026-10-02T16:20:04.202Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-17231fde85734485","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"RepositoryCheck::DispatchWorker","correlation_id":"5649ea4b414487c7fa75b50d7e0ae149","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b22e99a0acd71e0cf2962b3ba955d6f34621628300baf74d4c861b087bc17605","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:04.207Z","job_size_bytes":11,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::BatchWorker JID-7174cbaeca28896b5c8c1c17: start","job_status":"start","queue_duration_s":0.414603,"scheduling_latency_s":0.414618} gitlab # [ 177.926634] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:04.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"repository_check","args":["default"],"class":"RepositoryCheck::BatchWorker","jid":"7174cbaeca28896b5c8c1c17","created_at":"2026-10-02T16:20:04.202Z","trace_propagation_headers":{"sentry-trace":"f7bc3328c0274f7cbe89c5f7054981bc-17231fde85734485","baggage":"sentry-trace_id=f7bc3328c0274f7cbe89c5f7054981bc,sentry-release=Unknown"},"meta.caller_id":"RepositoryCheck::DispatchWorker","correlation_id":"5649ea4b414487c7fa75b50d7e0ae149","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b22e99a0acd71e0cf2962b3ba955d6f34621628300baf74d4c861b087bc17605","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:04.207Z","job_size_bytes":11,"pid":1644,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::BatchWorker JID-7174cbaeca28896b5c8c1c17: done: 0.02053 sec","job_status":"done","queue_duration_s":0.414603,"scheduling_latency_s":0.414618,"redis_calls":13,"redis_duration_s":0.002125,"redis_read_bytes":12,"redis_write_bytes":1387,"redis_cache_calls":1,"redis_cache_duration_s":0.000227,"redis_cache_read_bytes":1,"redis_cache_write_bytes":37,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000878,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":513,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006159721,"duration_s":0.02053,"completed_at":"2026-10-02T16:20:04.642Z","load_balancing_strategy":"primary","db_duration_s":0.004177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.181195] postgres[2456]: [2456] ERROR: relation "table_without_models" does not exist gitlab # [ 187.183145] postgres[2456]: [2456] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 190.891906] runuser[2245]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 190.894576] setup-gitlab-start[2231]: + systemctl restart gitlab-sidekiq.service gitlab # [ 190.929538] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 190.970698] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:20.170Z","message":"Shutting down"} gitlab # [ 190.972301] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:20.170Z","message":"Scheduler exiting..."} gitlab # [ 190.973472] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:20.170Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 190.974928] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:20.171Z","message":"Scheduler exiting..."} gitlab # [ 191.473393] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:20.673Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 195.978782] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:25.178Z","message":"Bye!"} gitlab # [ 196.009577] sidekiq-cluster[1644]: {"severity":"INFO","time":"2026-10-02T16:20:25.210Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1644,"worker_id":"sidekiq_0","memwd_rss_bytes":946917376,"retry":0} gitlab # [ 196.676219] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 196.678293] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 196.680165] systemd[1]: gitlab-sidekiq.service: Consumed 1min 967ms CPU time over 2min 854ms wall clock time, 958.2M memory peak, 4.4M read from disk. gitlab # [ 196.685382] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 196.689714] setup-gitlab-start[2231]: + mkdir -p /var/lib/secrets gitlab # [ 196.692914] setup-gitlab-start[2231]: + echo nixbot-test-token-of-20-chars gitlab # [ 196.694555] setup-gitlab-start[2231]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 196.697595] setup-gitlab-start[2231]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 197.380766] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:26 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 654 gitlab # [ 197.951145] gitaly[919]: time="2026-10-02T16:20:27.151Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.151" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.960132] gitaly[919]: time="2026-10-02T16:20:27.151Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.151" grpc.time_ms=0.314 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.097549] gitaly[919]: time="2026-10-02T16:20:27.298Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.157" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.297" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.120765] gitaly[919]: time="2026-10-02T16:20:27.298Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.157" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.297" grpc.time_ms=0.471 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.155722] gitaly[919]: time="2026-10-02T16:20:27.356Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH diskcache=34aa8640-dbbc-4ad2-9ff8-dfcb3fd3d23c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:20:37.162" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.303" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.173645] gitaly[919]: time="2026-10-02T16:20:27.357Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1575 command.oublock=32 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=16 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:20:37.162" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.303" grpc.time_ms=52.843 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.174746] gitaly[919]: time="2026-10-02T16:20:27.373Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.232" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.373" pid=919 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 198.179232] gitaly[919]: time="2026-10-02T16:20:27.373Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.232" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.373" grpc.time_ms=0.398 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.189675] gitaly[919]: time="2026-10-02T16:20:27.390Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:20:37.241" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.381" grpc.time_ms=8.249 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.241864] gitaly[919]: time="2026-10-02T16:20:27.442Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.442" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.253471] gitaly[919]: time="2026-10-02T16:20:27.442Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.442" grpc.time_ms=0.265 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.287073] gitaly[919]: time="2026-10-02T16:20:27.487Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH diskcache=d164db3a-84ff-4b69-971b-f5ae1e659849 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:20:37.004" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.446" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.291239] gitaly[919]: time="2026-10-02T16:20:27.487Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1577 command.oublock=32 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:20:37.004" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.446" grpc.time_ms=41.667 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.296367] gitaly[919]: time="2026-10-02T16:20:27.496Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.055" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.496" pid=919 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 198.298146] gitaly[919]: time="2026-10-02T16:20:27.498Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.055" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.496" grpc.time_ms=0.845 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.309085] gitaly[919]: time="2026-10-02T16:20:27.509Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:20:37.061" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.501" grpc.time_ms=7.404 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.335508] gitaly[919]: time="2026-10-02T16:20:27.536Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:20:37.094" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.535" grpc.time_ms=0.188 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.340664] gitaly[919]: time="2026-10-02T16:20:27.541Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:20:37.100" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:27.540" grpc.time_ms=0.43 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.842906] gitaly[919]: time="2026-10-02T16:20:28.043Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNFVBE7HGFQQ2DNMX3GVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:20:38.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:28.034" grpc.time_ms=8.635 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.157515] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:28 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 1753 gitlab # [ 199.163437] setup-gitlab-start[2525]: {"id":1,"message":null,"error":null} gitlab # [ 199.865241] sidekiq-cluster[2512]: {"severity":"INFO","time":"2026-10-02T16:20:29.048Z","message":"Starting cluster with 1 processes"} gitlab # [ 204.270356] gitaly[919]: time="2026-10-02T16:20:33.470Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNPFGX9J1AQ4H9AGHXK60 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:20:43.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:33.459" grpc.time_ms=11.318 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.346573] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:33 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 152 gitlab # [ 204.352641] setup-gitlab-start[2231]: + mkdir -p /tmp/test-flake gitlab # [ 204.354940] setup-gitlab-start[2231]: + cd /tmp/test-flake gitlab # [ 204.356702] setup-gitlab-start[2231]: + git init -b master gitlab # [ 204.376059] setup-gitlab-start[2600]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 204.378345] setup-gitlab-start[2231]: + git config user.name test gitlab # [ 204.381244] setup-gitlab-start[2231]: + git config user.email test@example.com gitlab # [ 204.383669] setup-gitlab-start[2231]: + cat gitlab # [ 204.386946] setup-gitlab-start[2231]: + git add flake.nix gitlab # [ 204.392454] setup-gitlab-start[2231]: + git commit -m 'initial commit' gitlab # [ 204.416412] setup-gitlab-start[2605]: [master (root-commit) 80d6afd] initial commit gitlab # [ 204.417593] setup-gitlab-start[2605]: 1 file changed, 10 insertions(+) gitlab # [ 204.419062] setup-gitlab-start[2605]: create mode 100644 flake.nix gitlab # [ 204.419905] setup-gitlab-start[2231]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 204.426552] setup-gitlab-start[2231]: + git push -u origin master gitlab # [ 204.494845] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:33 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 35 gitlab # [ 205.583055] gitaly[919]: time="2026-10-02T16:20:34.782Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNPS5XX9M72BHZS0QN5QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:34.769" grpc.time_ms=12.825 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 205.612863] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:34 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1081 gitlab # [ 205.883730] gitaly[919]: time="2026-10-02T16:20:35.084Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:20:45.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:35.072" grpc.time_ms=11.686 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.958376] gitaly[919]: time="2026-10-02T16:20:35.158Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T16:21:05.047" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T16:20:35.148" grpc.time_ms=10.537 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.018909] gitaly[919]: time="2026-10-02T16:20:35.219Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YPNQW9VV9PQ86T6618P9AW duration_ms=230ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 206.057572] gitaly[919]: time="2026-10-02T16:20:35.258Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YPNQW9VV9PQ86T6618P9AW duration_ms=37ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 206.059281] gitaly[919]: time="2026-10-02T16:20:35.258Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:20:34.988" grpc.time_ms=270.531 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 206.124130] gitaly[919]: time="2026-10-02T16:20:35.324Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:20:35.323" grpc.time_ms=0.936 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 206.184509] gitaly[919]: time="2026-10-02T16:20:35.385Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:20:35.384" grpc.time_ms=0.707 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 206.229366] gitaly[919]: time="2026-10-02T16:20:35.430Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:20:35.429" grpc.time_ms=0.842 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 206.357493] gitaly[919]: time="2026-10-02T16:20:35.558Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:20:45.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:35.549" grpc.time_ms=8.71 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.407348] gitaly[919]: time="2026-10-02T16:20:35.607Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T16:20:45.047" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:35.588" grpc.time_ms=19.395 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.432545] gitaly[919]: time="2026-10-02T16:20:35.633Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YPNQW9VV9PQ86T6618P9AW duration_ms=167ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 206.434173] gitaly[919]: time="2026-10-02T16:20:35.633Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:20:35.465" grpc.time_ms=168.719 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 206.458292] gitaly[919]: time="2026-10-02T16:20:35.658Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW diskcache=c5a8696e-7ac9-4c24-9171-e8939257dbb8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:34.909" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 206.459530] gitaly[919]: time="2026-10-02T16:20:35.659Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=201 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13625 command.oublock=32 command.real_time_ms=737 command.spawn_token_fork_ms=0 command.system_time_ms=119 command.user_time_ms=82 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:34.909" grpc.time_ms=749.434 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 206.480059] gitlab-workhorse[828]: time="2026-10-02T16:20:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 206.480957] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:35 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 855 gitlab # [ 206.486291] setup-gitlab-start[2609]: To http://gitlab/root/test-flake.git gitlab # [ 206.487360] setup-gitlab-start[2609]: * [new branch] master -> master gitlab # [ 206.492095] setup-gitlab-start[2609]: branch 'master' set up to track 'origin/master'. gitlab # [ 206.495518] systemd[1]: Finished setup-gitlab.service. gitlab # [ 206.502315] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 207.56 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 207.718371] nixbot[2696]: {"ts": "2026-10-02T16:20:36.918Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 207.754544] nixbot[2696]: {"ts": "2026-10-02T16:20:36.955Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 207.780126] nixbot[2696]: {"ts": "2026-10-02T16:20:36.980Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 207.784706] nixbot[2696]: {"ts": "2026-10-02T16:20:36.985Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 207.789224] nixbot[2696]: {"ts": "2026-10-02T16:20:36.989Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 207.797742] nixbot[2696]: {"ts": "2026-10-02T16:20:36.998Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 207.804501] nixbot[2696]: {"ts": "2026-10-02T16:20:37.004Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 207.807059] nixbot[2696]: {"ts": "2026-10-02T16:20:37.007Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 207.815896] nixbot[2696]: {"ts": "2026-10-02T16:20:37.016Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 207.820964] nixbot[2696]: {"ts": "2026-10-02T16:20:37.021Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 207.827070] nixbot[2696]: {"ts": "2026-10-02T16:20:37.027Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 207.833879] nixbot[2696]: {"ts": "2026-10-02T16:20:37.034Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 207.837859] nixbot[2696]: {"ts": "2026-10-02T16:20:37.038Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 207.840480] nixbot[2696]: {"ts": "2026-10-02T16:20:37.041Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 207.844569] nixbot[2696]: {"ts": "2026-10-02T16:20:37.045Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 207.847329] nixbot[2696]: {"ts": "2026-10-02T16:20:37.047Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 207.850325] nixbot[2696]: {"ts": "2026-10-02T16:20:37.050Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 207.852737] nixbot[2696]: {"ts": "2026-10-02T16:20:37.053Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 207.857701] nixbot[2696]: {"ts": "2026-10-02T16:20:37.058Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 207.860511] nixbot[2696]: {"ts": "2026-10-02T16:20:37.061Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 207.863598] nixbot[2696]: {"ts": "2026-10-02T16:20:37.064Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 207.871192] nixbot[2696]: {"ts": "2026-10-02T16:20:37.071Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 207.875741] nixbot[2696]: {"ts": "2026-10-02T16:20:37.076Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 207.881428] nixbot[2696]: {"ts": "2026-10-02T16:20:37.081Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 207.884437] nixbot[2696]: {"ts": "2026-10-02T16:20:37.084Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 207.889927] nixbot[2696]: {"ts": "2026-10-02T16:20:37.090Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 207.892860] nixbot[2696]: {"ts": "2026-10-02T16:20:37.093Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 207.897153] nixbot[2696]: {"ts": "2026-10-02T16:20:37.097Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 207.901492] nixbot[2696]: {"ts": "2026-10-02T16:20:37.102Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 207.916727] nixbot[2696]: {"ts": "2026-10-02T16:20:37.117Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 207.926053] nixbot[2696]: {"ts": "2026-10-02T16:20:37.126Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 207.929314] nixbot[2696]: {"ts": "2026-10-02T16:20:37.129Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 207.932571] nixbot[2696]: {"ts": "2026-10-02T16:20:37.133Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 207.940382] nixbot[2696]: {"ts": "2026-10-02T16:20:37.141Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 207.945476] nixbot[2696]: {"ts": "2026-10-02T16:20:37.146Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 207.953229] nixbot[2696]: {"ts": "2026-10-02T16:20:37.153Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 207.958332] nixbot[2696]: {"ts": "2026-10-02T16:20:37.158Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 208.345566] nixbot[2696]: INFO: Started server process [2696] gitlab # [ 208.347917] nixbot[2696]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 208.363636] nixbot[2696]: INFO: ::ffff:127.0.0.1:51016 - "GET /health HTTP/1.1" 200 OK gitlab # [ 208.371945] systemd[1]: Started nixbot CI service. gitlab # [ 208.376908] systemd[1]: Reached target Multi-User System. gitlab # [ 208.378939] systemd[1]: Startup finished in 1.205s (kernel) + 3.644s (initrd) + 3min 23.528s (userspace) = 3min 28.378s. gitlab # [ 208.582514] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:37 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 219 gitlab # [ 208.587826] nixbot[2696]: {"ts": "2026-10-02T16:20:37.784Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 208.601477] nixbot[2696]: {"ts": "2026-10-02T16:20:37.801Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 208.655534] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 45 gitlab # [ 208.660248] nixbot[2696]: {"ts": "2026-10-02T16:20:37.859Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 208.665598] nixbot[2696]: {"ts": "2026-10-02T16:20:37.861Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 208.845986] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 181 gitlab # [ 208.851630] nixbot[2696]: {"ts": "2026-10-02T16:20:38.047Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: (finished: waiting for unit nixbot.service, in 2.07 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 209.027156] nixbot[2696]: INFO: ::ffff:127.0.0.1:51018 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 209.129164] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 75 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 0.10 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 209.412937] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 557 gitlab # [ 209.418564] nixbot[2696]: {"ts": "2026-10-02T16:20:38.614Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 209.466285] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 42 gitlab # [ 209.471404] nixbot[2696]: {"ts": "2026-10-02T16:20:38.667Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 209.536795] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:38 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 66 gitlab # [ 209.541893] nixbot[2696]: {"ts": "2026-10-02T16:20:38.738Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 209.600273] gitaly[919]: time="2026-10-02T16:20:38.800Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNVPT71813A35HQXMBX8A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-02T16:21:08.690" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:38.791" grpc.time_ms=8.519 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.652060] gitaly[919]: time="2026-10-02T16:20:38.852Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNVPT71813A35HQXMBX8A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:08.750" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:20:38.850" grpc.time_ms=1.17 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.694220] gitaly[919]: time="2026-10-02T16:20:38.894Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNVPT71813A35HQXMBX8A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-02T16:20:48.002" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:20:38.882" grpc.time_ms=11.495 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.731203] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 183 gitlab # [ 209.734449] nixbot[2696]: {"ts": "2026-10-02T16:20:38.934Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 209.840980] nixbot[2696]: {"ts": "2026-10-02T16:20:39.041Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 209.847210] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 99 gitlab # [ 209.858627] nixbot[2696]: {"ts": "2026-10-02T16:20:39.058Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 209.975203] gitaly[919]: time="2026-10-02T16:20:39.175Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNW2DD0PM58DECCCB11BV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.168" grpc.time_ms=6.744 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.003770] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 86 gitlab # [ 210.113145] gitaly[919]: time="2026-10-02T16:20:39.313Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNW59NS81T10BKFPF1R5G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.300" pid=919 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 210.115268] gitaly[919]: time="2026-10-02T16:20:39.313Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNW59NS81T10BKFPF1R5G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.300" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.160993] gitaly[919]: time="2026-10-02T16:20:39.361Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNW59NS81T10BKFPF1R5G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.300" grpc.time_ms=60.889 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.162576] gitlab-workhorse[828]: time="2026-10-02T16:20:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 210.164211] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 152 gitlab # [ 210.256292] gitaly[919]: time="2026-10-02T16:20:39.456Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWASC3ENPFJS89BNV627 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.438" pid=919 remote_ip="2001:db8:1::1" request_sha=ef3b8671d45f59178c339c53410e5dbd43b5b467 response_bytes=223 span.kind=server system=grpc gitlab # [ 210.257476] gitaly[919]: time="2026-10-02T16:20:39.456Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1208 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWASC3ENPFJS89BNV627 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.438" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.258542] gitlab-workhorse[828]: time="2026-10-02T16:20:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 210.259281] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 71 gitlab # [ 210.260113] gitaly[919]: time="2026-10-02T16:20:39.457Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1208 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWASC3ENPFJS89BNV627 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.438" grpc.time_ms=18.773 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.392142] gitaly[919]: time="2026-10-02T16:20:39.592Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNWFJV868G07TJYGDRWJD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.580" grpc.time_ms=12.023 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.394265] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 54 gitlab # [ 210.478100] gitaly[919]: time="2026-10-02T16:20:39.678Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWJ7G2PBNSBR86KAPGGB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.664" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 210.499401] gitaly[919]: time="2026-10-02T16:20:39.678Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWJ7G2PBNSBR86KAPGGB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.664" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.519399] gitaly[919]: time="2026-10-02T16:20:39.720Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWJ7G2PBNSBR86KAPGGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.664" grpc.time_ms=54.906 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.520170] gitlab-workhorse[828]: time="2026-10-02T16:20:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 210.521606] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 97 gitlab # [ 210.644243] gitaly[919]: time="2026-10-02T16:20:39.844Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNWQ3HZQGYDCPC74H58HX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.832" grpc.time_ms=11.655 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.646199] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 67 gitlab # [ 210.770872] gitaly[919]: time="2026-10-02T16:20:39.970Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWT667T0XB0KC824B8J2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.947" pid=919 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 210.772638] gitaly[919]: time="2026-10-02T16:20:39.970Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWT667T0XB0KC824B8J2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.947" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.774767] gitaly[919]: time="2026-10-02T16:20:39.971Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNWT667T0XB0KC824B8J2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:39.947" grpc.time_ms=23.561 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.833268] gitlab-workhorse[828]: time="2026-10-02T16:20:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 210.833822] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 154 gitlab # [ 211.018079] gitaly[919]: time="2026-10-02T16:20:40.218Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:10.017" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:20:40.217" grpc.time_ms=0.816 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.086075] gitaly[919]: time="2026-10-02T16:20:40.286Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T16:21:10.077" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:40.277" grpc.time_ms=8.664 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.147814] gitaly[919]: time="2026-10-02T16:20:40.348Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:20:50.195" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:40.336" grpc.time_ms=11.902 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.200489] gitaly[919]: time="2026-10-02T16:20:40.401Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:20:50.248" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:40.389" grpc.time_ms=11.688 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.312294] gitaly[919]: time="2026-10-02T16:20:40.512Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:20:50.071" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:40.512" grpc.time_ms=0.378 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.360067] gitaly[919]: time="2026-10-02T16:20:40.560Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:20:50.107" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:40.547" grpc.time_ms=12.408 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.409198] gitaly[919]: time="2026-10-02T16:20:40.609Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:20:40.608" grpc.time_ms=0.847 pid=919 span.kind=server system=grpc gitlab # [ 211.462922] gitaly[919]: time="2026-10-02T16:20:40.663Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:20:40.662" grpc.time_ms=0.664 pid=919 span.kind=server system=grpc gitlab # [ 211.483734] gitaly[919]: time="2026-10-02T16:20:40.684Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW diskcache=f43b98a8-181f-44d5-aaf5-24f89fe52619 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:20:50.124" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:40.564" pid=919 ref=refs/keep-around/80d6afdddfb9366d3eeba0eaf297ab097f89953c remote_ip="2001:db8:1::1" revision=80d6afdddfb9366d3eeba0eaf297ab097f89953c span.kind=server system=grpc username=root gitlab # [ 211.484620] gitaly[919]: time="2026-10-02T16:20:40.684Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=101 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5343 command.oublock=8 command.real_time_ms=109 command.spawn_token_fork_ms=0 command.system_time_ms=67 command.user_time_ms=34 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:20:50.124" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:20:40.564" grpc.time_ms=119.892 pid=919 ref=refs/keep-around/80d6afdddfb9366d3eeba0eaf297ab097f89953c remote_ip="2001:db8:1::1" revision=80d6afdddfb9366d3eeba0eaf297ab097f89953c span.kind=server system=grpc username=root gitlab # [ 211.510504] gitaly[919]: time="2026-10-02T16:20:40.711Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:10.510" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:20:40.710" grpc.time_ms=0.476 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.703419] nixbot[2696]: {"ts": "2026-10-02T16:20:40.903Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c \"HTTP/1.1 201 Created\""} gitlab # [ 211.709970] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 830 gitlab # [ 211.720849] nixbot[2696]: {"ts": "2026-10-02T16:20:40.921Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-80d6afdddfb9-0"} gitlab # [ 211.915583] nixbot[2696]: {"ts": "2026-10-02T16:20:41.116Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-80d6afdddfb9-0", "argv0": "bwrap"} gitlab # [ 212.021125] systemd[1]: Started Nix Daemon. gitlab # [ 212.100782] nix-daemon[2866]: accepted connection from pid 2858, user nixbot gitlab # [ 217.126111] nix-daemon[2866]: accepted connection from pid 2912, user nixbot gitlab # [ 217.139938] nix-daemon[2866]: accepted connection from pid 2858, user nixbot gitlab # [ 221.880346] nix-daemon[2866]: accepted connection from pid 2858, user nixbot gitlab # [ 221.985543] nix-daemon[2866]: accepted connection from pid 2961, user nixbot gitlab # [ 222.032517] gitaly[919]: time="2026-10-02T16:20:51.232Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPP7V77QRN1AYGQD4K0SFH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:21.025" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:20:51.226" grpc.time_ms=6.452 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.082267] gitaly[919]: time="2026-10-02T16:20:51.282Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPP7V77QRN1AYGQD4K0SFH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:01.134" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:51.274" grpc.time_ms=7.646 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.128285] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 152 gitlab # [ 222.134751] nixbot[2696]: {"ts": "2026-10-02T16:20:51.334Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c \"HTTP/1.1 201 Created\""} gitlab # [ 222.213582] gitaly[919]: time="2026-10-02T16:20:51.414Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPP80CZ2XEVF5XYP6HHG97 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:21.312" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:20:51.413" grpc.time_ms=0.767 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.386629] gitaly[919]: time="2026-10-02T16:20:51.587Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPP80CZ2XEVF5XYP6HHG97 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:01.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:51.574" grpc.time_ms=12.55 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.468512] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 328 gitlab # [ 222.470487] nixbot[2696]: {"ts": "2026-10-02T16:20:51.671Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c \"HTTP/1.1 201 Created\""} gitlab # [ 225.399105] sidekiq-cluster[2573]: 2026-10-02T16:20:54.598Z pid=2573 tid=xb9 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2573, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>449695744} gitlab # [ 225.571642] sidekiq-cluster[2573]: W, [2026-10-02T16:20:54.772128 #2573] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 225.610168] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:20:54.810Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 226.148162] nix-daemon[2866]: accepted connection from pid 3033, user nixbot gitlab # [ 226.236969] gitaly[919]: time="2026-10-02T16:20:55.437Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPPBYS3NPE490N4921HRBB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:25.335" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:20:55.436" grpc.time_ms=0.975 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 226.285934] gitaly[919]: time="2026-10-02T16:20:55.486Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPPBYS3NPE490N4921HRBB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:05.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:20:55.477" grpc.time_ms=8.708 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 226.356382] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 169 gitlab # [ 226.363610] nixbot[2696]: {"ts": "2026-10-02T16:20:55.557Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c \"HTTP/1.1 201 Created\""} gitlab # [ 226.768629] gitaly[919]: time="2026-10-02T16:20:55.969Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPPCFB8778J3MRM63RNG1Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:55.956" grpc.time_ms=12.353 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 226.770058] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:55 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 54 gitlab # [ 226.860320] gitaly[919]: time="2026-10-02T16:20:56.061Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPPCH56V0MPJX937JQK5YM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:56.054" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 226.860833] gitaly[919]: time="2026-10-02T16:20:56.061Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPPCH56V0MPJX937JQK5YM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:56.054" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 226.861143] gitaly[919]: time="2026-10-02T16:20:56.061Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPPCH56V0MPJX937JQK5YM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:56.054" grpc.time_ms=7.018 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 226.861750] gitlab-workhorse[828]: time="2026-10-02T16:20:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 226.862601] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 89 gitlab # [ 226.963187] gitaly[919]: time="2026-10-02T16:20:56.163Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPPCNQ2NA3TP6P5PMGSMYD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:56.156" grpc.time_ms=6.841 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 226.964642] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 44 gitlab # [ 227.008193] gitaly[919]: time="2026-10-02T16:20:56.208Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPPCQ6A6XSSY7P0HVNVK0T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:56.201" pid=919 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 227.008968] gitaly[919]: time="2026-10-02T16:20:56.209Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPPCQ6A6XSSY7P0HVNVK0T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:56.201" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 227.010227] gitaly[919]: time="2026-10-02T16:20:56.210Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPPCQ6A6XSSY7P0HVNVK0T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:20:56.201" grpc.time_ms=8.886 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 227.011578] gitlab-workhorse[828]: time="2026-10-02T16:20:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 227.012779] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:20:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 46 gitlab # [ 227.399138] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:20:56.598Z","message":"GitLab reliable fetch activated!"} gitlab # [ 227.401315] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:20:56.598Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 248.349948] postgres[3258]: [3258] ERROR: relation "table_without_models" does not exist gitlab # [ 248.351891] postgres[3258]: [3258] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 250.791948] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:19.991Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 250.794769] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:19.995Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 250.796243] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:19.995Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 250.797433] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:19.995Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 250.798567] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:19.996Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 42.66 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 251.886133] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:21 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 40 gitlab # [ 251.985150] gitaly[919]: time="2026-10-02T16:21:21.185Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ524J4ZEP54BDS5K4W8F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:21.175" grpc.time_ms=9.842 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 251.986377] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:21 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 93 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 252.276103] gitaly[919]: time="2026-10-02T16:21:21.476Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T16:21:51.267" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T16:21:21.468" grpc.time_ms=8.219 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.313678] gitaly[919]: time="2026-10-02T16:21:21.514Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 duration_ms=120ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 252.347341] gitaly[919]: time="2026-10-02T16:21:21.547Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 duration_ms=33ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 252.350133] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:21.548Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"44a95521fa4a3b2449aaf0c1","created_at":"2026-10-02T16:20:27.401Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.426Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-44a95521fa4a3b2449aaf0c1: start","job_status":"start","queue_duration_s":54.121205,"scheduling_latency_s":54.121289} gitlab # [ 252.364160] gitaly[919]: time="2026-10-02T16:21:21.564Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:21.391" grpc.time_ms=156.495 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 252.399679] gitaly[919]: time="2026-10-02T16:21:21.600Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:21.598" grpc.time_ms=1.164 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 252.435093] gitaly[919]: time="2026-10-02T16:21:21.635Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:21.634" grpc.time_ms=0.722 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 252.464703] gitaly[919]: time="2026-10-02T16:21:21.665Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:21.664" grpc.time_ms=0.542 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 252.495499] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:21.695Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"44a95521fa4a3b2449aaf0c1","created_at":"2026-10-02T16:20:27.401Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.426Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-44a95521fa4a3b2449aaf0c1: done: 0.14827 sec","job_status":"done","queue_duration_s":54.121205,"scheduling_latency_s":54.121289,"redis_calls":24,"redis_duration_s":0.0042439999999999995,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.002202,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000792,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.070332682,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.14827,"completed_at":"2026-10-02T16:21:21.695Z","load_balancing_strategy":"primary","db_duration_s":0.035181,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.496883] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:21.697Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"1b491c818a86bb53eac29fae","created_at":"2026-10-02T16:20:27.434Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.437Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-1b491c818a86bb53eac29fae: start","job_status":"start","queue_duration_s":54.259138,"scheduling_latency_s":54.259155} gitlab # [ 252.572234] gitaly[919]: time="2026-10-02T16:21:21.772Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 duration_ms=78ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 252.573309] gitaly[919]: time="2026-10-02T16:21:21.773Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:21.694" grpc.time_ms=78.809 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 252.581230] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:21.781Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"1b491c818a86bb53eac29fae","created_at":"2026-10-02T16:20:27.434Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.437Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-1b491c818a86bb53eac29fae: done: 0.08446 sec","job_status":"done","queue_duration_s":54.259138,"scheduling_latency_s":54.259155,"redis_calls":14,"redis_duration_s":0.002613,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000385,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001283,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06242218,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.08446,"completed_at":"2026-10-02T16:21:21.781Z","load_balancing_strategy":"primary","db_duration_s":0.017081,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.584900] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:21.782Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"6716a37cd9f9da73042c5297","created_at":"2026-10-02T16:20:27.915Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.924Z","job_size_bytes":5,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-6716a37cd9f9da73042c5297: start","job_status":"start","queue_duration_s":53.858436,"scheduling_latency_s":53.858453} gitlab # [ 252.655148] gitaly[919]: time="2026-10-02T16:21:21.855Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 diskcache=57fd61d7-6018-4323-ad8b-3743e1b5fab7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:21.313" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 252.656685] gitlab-workhorse[828]: time="2026-10-02T16:21:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 252.657711] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:21 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 614 gitlab # To http://gitlab/root/test-flake.git gitlab # 80d6afd..763551d master -> master[ 252.669080] gitaly[919]: time="2026-10-02T16:21:21.868Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=191 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13196 command.oublock=32 command.real_time_ms=530 command.spawn_token_fork_ms=1 command.system_time_ms=120 command.user_time_ms=71 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:21.313" grpc.time_ms=542.259 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.90 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45' gitlab # [ 252.711948] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:21.912Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"6716a37cd9f9da73042c5297","created_at":"2026-10-02T16:20:27.915Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.924Z","job_size_bytes":5,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-6716a37cd9f9da73042c5297: done: 0.129384 sec","job_status":"done","queue_duration_s":53.858436,"scheduling_latency_s":53.858453,"redis_calls":12,"redis_duration_s":0.00191,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000250591,"duration_s":0.129384,"completed_at":"2026-10-02T16:21:21.912Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017056,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.715317] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:21.914Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"ae04bfdd8d188a10f73b525f","created_at":"2026-10-02T16:20:27.966Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.973Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-ae04bfdd8d188a10f73b525f: start","job_status":"start","queue_duration_s":53.941399,"scheduling_latency_s":53.941417} gitlab # [ 252.756219] nixbot[2696]: INFO: ::ffff:127.0.0.1:32852 - "GET /api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45', in 0.07 seconds) gitlab # [ 252.852943] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:22.053Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"ae04bfdd8d188a10f73b525f","created_at":"2026-10-02T16:20:27.966Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.973Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-ae04bfdd8d188a10f73b525f: done: 0.13851 sec","job_status":"done","queue_duration_s":53.941399,"scheduling_latency_s":53.941417,"redis_calls":11,"redis_duration_s":0.002017,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000454,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0004,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.107,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.107,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000298082,"duration_s":0.13851,"completed_at":"2026-10-02T16:21:22.053Z","load_balancing_strategy":"primary","db_duration_s":0.043179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.922144] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:22.122Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"0ea34543f8b25f4cf2f8c210","created_at":"2026-10-02T16:20:27.977Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.985Z","job_size_bytes":89,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-0ea34543f8b25f4cf2f8c210: start","job_status":"start","queue_duration_s":54.136931,"scheduling_latency_s":54.136981} gitlab # [ 252.976363] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:22.176Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"0ea34543f8b25f4cf2f8c210","created_at":"2026-10-02T16:20:27.977Z","trace_propagation_headers":{"sentry-trace":"0582d58af785490da6285a8452405e39-1308865e51ff43f4","baggage":"sentry-trace_id=0582d58af785490da6285a8452405e39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNFVBE7HGFQQ2DNMX3GVH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:27.985Z","job_size_bytes":89,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-0ea34543f8b25f4cf2f8c210: done: 0.054645 sec","job_status":"done","queue_duration_s":54.136931,"scheduling_latency_s":54.136981,"redis_calls":11,"redis_duration_s":0.003577,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001003,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000691,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000509841,"duration_s":0.054645,"completed_at":"2026-10-02T16:21:22.176Z","load_balancing_strategy":"primary","db_duration_s":0.006963,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.029155] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:22.229Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 80d6afdddfb9366d3eeba0eaf297ab097f89953c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"5c750e9b941235c0376f7f37","created_at":"2026-10-02T16:20:35.497Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-6d0f77a0c5364ac5","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:35.518Z","job_size_bytes":185,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-5c750e9b941235c0376f7f37: start","job_status":"start","queue_duration_s":46.710194,"scheduling_latency_s":46.710238} gitlab # [ 253.054132] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:22.254Z","message":"changes: \"0000000000000000000000000000000000000000 80d6afdddfb9366d3eeba0eaf297ab097f89953c refs/heads/master\\n\""} gitlab # [ 253.108867] gitaly[919]: time="2026-10-02T16:21:22.308Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:32.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:22.296" grpc.time_ms=11.815 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.125481] gitaly[919]: time="2026-10-02T16:21:22.326Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:21:32.024" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:22.315" grpc.time_ms=10.524 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.202287] gitaly[919]: time="2026-10-02T16:21:22.402Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T16:21:32.098" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:22.389" grpc.time_ms=13.512 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.283198] gitaly[919]: time="2026-10-02T16:21:22.483Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3931 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:22.482" grpc.time_ms=0.73 pid=919 span.kind=server system=grpc gitlab # [ 253.334470] gitaly[919]: time="2026-10-02T16:21:22.535Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:22.534" grpc.time_ms=0.431 pid=919 span.kind=server system=grpc gitlab # [ 253.365727] gitaly[919]: time="2026-10-02T16:21:22.566Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW diskcache=d25fe7c3-02df-40a8-9ec1-ae098f1a3ad9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:21:32.149" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:22.440" pid=919 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 253.367045] gitaly[919]: time="2026-10-02T16:21:22.566Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=128 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5129 command.oublock=8 command.real_time_ms=115 command.spawn_token_fork_ms=0 command.system_time_ms=95 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:21:32.149" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:22.440" grpc.time_ms=126.202 pid=919 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 253.387739] gitaly[919]: time="2026-10-02T16:21:22.588Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:21:32.297" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:22.587" grpc.time_ms=0.239 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.404166] gitaly[919]: time="2026-10-02T16:21:22.604Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T16:21:32.002" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:22.593" grpc.time_ms=11.531 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.483586] gitaly[919]: time="2026-10-02T16:21:22.684Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=340 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-02T16:21:52.077" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:22.678" grpc.time_ms=5.992 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.520093] gitaly[919]: time="2026-10-02T16:21:22.720Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:21:32.129" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:22.719" grpc.time_ms=0.506 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.546420] gitaly[919]: time="2026-10-02T16:21:22.747Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T16:21:32.148" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T16:21:22.738" grpc.time_ms=8.058 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.569376] gitaly[919]: time="2026-10-02T16:21:22.770Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:32.171" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:22.761" grpc.time_ms=8.449 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.586184] gitaly[919]: time="2026-10-02T16:21:22.786Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:32.185" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:22.775" grpc.time_ms=11.091 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.616899] gitaly[919]: time="2026-10-02T16:21:22.817Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:21:52.203" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:22.803" grpc.time_ms=13.541 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45' gitlab # [ 253.811594] nixbot[2696]: INFO: ::ffff:127.0.0.1:56658 - "GET /api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45', in 0.05 seconds) gitlab # [ 254.030148] gitaly[919]: time="2026-10-02T16:21:23.230Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T16:21:33.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:23.228" grpc.time_ms=1.794 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.077169] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:23.276Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 80d6afdddfb9366d3eeba0eaf297ab097f89953c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"5c750e9b941235c0376f7f37","created_at":"2026-10-02T16:20:35.497Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-6d0f77a0c5364ac5","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:20:35.518Z","job_size_bytes":185,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-5c750e9b941235c0376f7f37: done: 1.047241 sec","job_status":"done","queue_duration_s":46.710194,"scheduling_latency_s":46.710238,"gitaly_calls":13,"gitaly_duration_s":0.300089,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.076326,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.004756,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008651,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002276,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004283,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.044833,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.011527,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.204,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.227,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.058,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.601044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000838374,"exclusive_lock_hold_duration_s":0.038931205,"duration_s":1.047241,"completed_at":"2026-10-02T16:21:23.276Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.100125,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 254.107724] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:23.279Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"90e07891e3e5d8f8a84dc27e","created_at":"2026-10-02T16:20:40.728Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-c6ba8f2cc1dc472d","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:40.737Z","job_size_bytes":65,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-90e07891e3e5d8f8a84dc27e: start","job_status":"start","queue_duration_s":42.542154,"scheduling_latency_s":42.542197} gitlab # [ 254.193157] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:23.393Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"90e07891e3e5d8f8a84dc27e","created_at":"2026-10-02T16:20:40.728Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-c6ba8f2cc1dc472d","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:40.737Z","job_size_bytes":65,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-90e07891e3e5d8f8a84dc27e: done: 0.113735 sec","job_status":"done","queue_duration_s":42.542154,"scheduling_latency_s":42.542197,"redis_calls":16,"redis_duration_s":0.00528,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002723,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000453689,"duration_s":0.113735,"completed_at":"2026-10-02T16:21:23.392Z","load_balancing_strategy":"primary","db_duration_s":0.020851,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 254.195659] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:23.396Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6671f201eb5c97a9477deea9","created_at":"2026-10-02T16:20:40.738Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-c6ba8f2cc1dc472d","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:40.742Z","job_size_bytes":65,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6671f201eb5c97a9477deea9: start","job_status":"start","queue_duration_s":42.653007,"scheduling_latency_s":42.653041} gitlab # [ 254.389325] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:23.589Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6671f201eb5c97a9477deea9","created_at":"2026-10-02T16:20:40.738Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-c6ba8f2cc1dc472d","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:40.742Z","job_size_bytes":65,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6671f201eb5c97a9477deea9: done: 0.193673 sec","job_status":"done","queue_duration_s":42.653007,"scheduling_latency_s":42.653041,"redis_calls":21,"redis_duration_s":0.006925,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.00344,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030814,"duration_s":0.193673,"completed_at":"2026-10-02T16:21:23.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.392759] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:23.591Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"69815f367a2795cb5a671529","created_at":"2026-10-02T16:20:40.865Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-c6ba8f2cc1dc472d","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:40.874Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-69815f367a2795cb5a671529: start","job_status":"start","queue_duration_s":42.716408,"scheduling_latency_s":42.716423} gitlab # [ 254.533368] gitaly[919]: time="2026-10-02T16:21:23.733Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:23.732" grpc.time_ms=0.688 pid=919 span.kind=server system=grpc gitlab # [ 254.565912] gitaly[919]: time="2026-10-02T16:21:23.766Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:23.765" grpc.time_ms=0.469 pid=919 span.kind=server system=grpc gitlab # [ 254.616085] gitaly[919]: time="2026-10-02T16:21:23.816Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:23.815" grpc.time_ms=0.618 pid=919 span.kind=server system=grpc gitlab # [ 254.638335] gitaly[919]: time="2026-10-02T16:21:23.839Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW diskcache=47489dee-c370-4fdd-b460-486b9bb24e2f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:21:53.095" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:23.696" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.638896] gitaly[919]: time="2026-10-02T16:21:23.839Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=134 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7580 command.oublock=0 command.real_time_ms=130 command.spawn_token_fork_ms=0 command.system_time_ms=82 command.user_time_ms=52 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:21:53.095" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:23.696" grpc.time_ms=142.886 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.748323] gitaly[919]: time="2026-10-02T16:21:23.948Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:21:53.347" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:23.947" grpc.time_ms=1.254 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.778646] gitaly[919]: time="2026-10-02T16:21:23.979Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNX067G5VTC6YF1RJGJGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:53.377" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:23.978" grpc.time_ms=0.854 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.797943] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:23.998Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"69815f367a2795cb5a671529","created_at":"2026-10-02T16:20:40.865Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-c6ba8f2cc1dc472d","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:40.874Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2573,"job_status":"rescheduled","message":"PipelineProcessWorker JID-69815f367a2795cb5a671529: rescheduled"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45' gitlab # [ 254.812579] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.012Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"69815f367a2795cb5a671529","created_at":"2026-10-02T16:20:40.865Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-c6ba8f2cc1dc472d","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:20:40.874Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-69815f367a2795cb5a671529: done: 0.421514 sec","job_status":"done","queue_duration_s":42.716408,"scheduling_latency_s":42.716423,"gitaly_calls":3,"gitaly_duration_s":0.17335,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.027525,"redis_read_bytes":460,"redis_write_bytes":29291,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001428,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001084,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.00976,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004083,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15578,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006294,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00053,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000551,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002367,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.022,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.154179,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.360481849,"duration_s":0.421514,"completed_at":"2026-10-02T16:21:24.012Z","load_balancing_strategy":"primary","db_duration_s":0.073949,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 254.825767] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.015Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","80d6afdddfb9366d3eeba0eaf297ab097f89953c 763551d41e8de88b31a5ee89e2050c071ec59b45 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"83f082ac63feca34a2663f03","created_at":"2026-10-02T16:21:21.756Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-5d6f7db5d92b4169","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:21.758Z","job_size_bytes":185,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-83f082ac63feca34a2663f03: start","job_status":"start","queue_duration_s":2.256926,"scheduling_latency_s":2.25696} gitlab # [ 254.879069] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.029Z","message":"changes: \"80d6afdddfb9366d3eeba0eaf297ab097f89953c 763551d41e8de88b31a5ee89e2050c071ec59b45 refs/heads/master\\n\""} gitlab # [ 254.888188] gitaly[919]: time="2026-10-02T16:21:24.087Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:34.040" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:24.081" grpc.time_ms=6.022 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45', in 0.08 seconds) gitlab # [ 254.897894] nixbot[2696]: INFO: ::ffff:127.0.0.1:56670 - "GET /api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 200 OK gitlab # [ 254.917360] gitaly[919]: time="2026-10-02T16:21:24.118Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:21:34.067" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:24.108" grpc.time_ms=9.767 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.933479] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.133Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"00cd714c3eb53225528358c4","created_at":"2026-10-02T16:21:24.132Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-3f82ebd9562349fc","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"fd9c21c2af524f0892271c6e","job_size_bytes":46,"pid":2573,"job_status":"deduplicated","message":"ProjectCacheWorker JID-00cd714c3eb53225528358c4: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 254.950472] gitaly[919]: time="2026-10-02T16:21:24.151Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T16:21:34.105" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:24.145" grpc.time_ms=5.603 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.964761] gitaly[919]: time="2026-10-02T16:21:24.165Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=657 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-02T16:21:54.053" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.154" grpc.time_ms=11.229 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.980891] gitaly[919]: time="2026-10-02T16:21:24.181Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T16:21:34.137" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T16:21:24.177" grpc.time_ms=3.769 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.001851] gitaly[919]: time="2026-10-02T16:21:24.202Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:34.153" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:24.193" grpc.time_ms=8.437 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.066853] gitaly[919]: time="2026-10-02T16:21:24.267Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T16:21:34.045" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.265" grpc.time_ms=1.613 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.097773] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.297Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","80d6afdddfb9366d3eeba0eaf297ab097f89953c 763551d41e8de88b31a5ee89e2050c071ec59b45 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"83f082ac63feca34a2663f03","created_at":"2026-10-02T16:21:21.756Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-5d6f7db5d92b4169","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:21.758Z","job_size_bytes":185,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-83f082ac63feca34a2663f03: done: 0.282374 sec","job_status":"done","queue_duration_s":2.256926,"scheduling_latency_s":2.25696,"gitaly_calls":7,"gitaly_duration_s":0.127269,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.033918000000000004,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.002277,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001169,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00337,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.020274,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006828,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.019,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.019,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000436089,"exclusive_lock_hold_duration_s":0.063263627,"duration_s":0.282374,"completed_at":"2026-10-02T16:21:24.297Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024527,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 255.102977] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.298Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"fd9c21c2af524f0892271c6e","created_at":"2026-10-02T16:21:22.330Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:22.356Z","job_size_bytes":46,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-fd9c21c2af524f0892271c6e: start","job_status":"start","queue_duration_s":1.942752,"scheduling_latency_s":1.942772} gitlab # [ 255.143917] gitaly[919]: time="2026-10-02T16:21:24.337Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-02T22:21:24.331" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:24.331" grpc.time_ms=4.868 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.153421] gitaly[919]: time="2026-10-02T16:21:24.354Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T16:21:54.242" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.343" grpc.time_ms=10.603 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.182682] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.383Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"fd9c21c2af524f0892271c6e","created_at":"2026-10-02T16:21:22.330Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:22.356Z","job_size_bytes":46,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-fd9c21c2af524f0892271c6e: done: 0.084138 sec","job_status":"done","queue_duration_s":1.942752,"scheduling_latency_s":1.942772,"gitaly_calls":2,"gitaly_duration_s":0.020915,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0069759999999999996,"redis_read_bytes":69,"redis_write_bytes":4882,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001336,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000553,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001576,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00284,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000671,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000283835,"duration_s":0.084138,"completed_at":"2026-10-02T16:21:24.383Z","load_balancing_strategy":"primary","db_duration_s":0.013881,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 255.186942] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.384Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"5add38727fe95c6d6eaee018","created_at":"2026-10-02T16:21:22.376Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:22.383Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-5add38727fe95c6d6eaee018: start","job_status":"start","queue_duration_s":2.001177,"scheduling_latency_s":2.001192} gitlab # [ 255.297599] gitaly[919]: time="2026-10-02T16:21:24.498Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW diskcache=082073c9-04d0-4c8b-bbd2-de3249738143 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:21:24.467" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.467" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.298878] gitaly[919]: time="2026-10-02T16:21:24.498Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1282 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:21:24.467" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.467" grpc.time_ms=30.699 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.346552] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.546Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"5add38727fe95c6d6eaee018","created_at":"2026-10-02T16:21:22.376Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:22.383Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-5add38727fe95c6d6eaee018: done: 0.161807 sec","job_status":"done","queue_duration_s":2.001177,"scheduling_latency_s":2.001192,"gitaly_calls":1,"gitaly_duration_s":0.034292,"redis_calls":15,"redis_duration_s":0.004567,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000616,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001576,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000904,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001471,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.080829267,"duration_s":0.161807,"completed_at":"2026-10-02T16:21:24.546Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.073263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.349072] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.549Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"b0050ff5cb56645dfdd186af","created_at":"2026-10-02T16:21:22.581Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:22.584Z","job_size_bytes":23,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b0050ff5cb56645dfdd186af: start","job_status":"start","queue_duration_s":1.964881,"scheduling_latency_s":1.964908} gitlab # [ 255.393595] gitaly[919]: time="2026-10-02T16:21:24.585Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T16:21:54.473" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.573" grpc.time_ms=11.503 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.424832] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.625Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"b0050ff5cb56645dfdd186af","created_at":"2026-10-02T16:21:22.581Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:22.584Z","job_size_bytes":23,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b0050ff5cb56645dfdd186af: done: 0.076073 sec","job_status":"done","queue_duration_s":1.964881,"scheduling_latency_s":1.964908,"gitaly_calls":1,"gitaly_duration_s":0.04449,"redis_calls":16,"redis_duration_s":0.004135,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.000332,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001886,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000791,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076073,"completed_at":"2026-10-02T16:21:24.625Z","load_balancing_strategy":"primary","db_duration_s":0.003161,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 255.426713] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:24.626Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"cc9e8661d9a57e9d1bbe8c4b","created_at":"2026-10-02T16:21:23.087Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:523e9c84ec2e71edb00d9793a8a32df31bc4ec4877f22861c08aa196e55c9de5","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.094Z","job_size_bytes":467,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-cc9e8661d9a57e9d1bbe8c4b: start","job_status":"start","queue_duration_s":1.531828,"scheduling_latency_s":1.531854} gitlab # [ 255.507681] gitaly[919]: time="2026-10-02T16:21:24.708Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:54.606" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.706" grpc.time_ms=0.988 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.536885] gitaly[919]: time="2026-10-02T16:21:24.736Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:21:34.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:24.724" grpc.time_ms=12.105 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.683904] gitaly[919]: time="2026-10-02T16:21:24.884Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:21:54.782" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.883" grpc.time_ms=1.191 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.718899] gitaly[919]: time="2026-10-02T16:21:24.919Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=59 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T16:21:54.815" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:24.915" grpc.time_ms=3.574 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.823923] gitaly[919]: time="2026-10-02T16:21:25.024Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:35.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:25.012" grpc.time_ms=12.008 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45' gitlab # [ 255.947186] nixbot[2696]: INFO: ::ffff:127.0.0.1:56680 - "GET /api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45', in 0.05 seconds) gitlab # [ 256.224710] gitaly[919]: time="2026-10-02T16:21:25.425Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:21:55.307" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:25.407" grpc.time_ms=17.358 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.277604] gitaly[919]: time="2026-10-02T16:21:25.478Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPNQW9VV9PQ86T6618P9AW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T16:21:55.361" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:25.461" grpc.time_ms=15.99 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.339468] sidekiq-cluster[2573]: {"severity":"WARN","time":"2026-10-02T16:21:25.539Z","class":"CreatePipelineWorker","correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"80d6afdddfb9366d3eeba0eaf297ab097f89953c","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"80d6afdddfb9366d3eeba0eaf297ab097f89953c","ref":"refs/heads/master"},"retry":0} gitlab # [ 256.349973] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.549Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"cc9e8661d9a57e9d1bbe8c4b","created_at":"2026-10-02T16:21:23.087Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:523e9c84ec2e71edb00d9793a8a32df31bc4ec4877f22861c08aa196e55c9de5","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.094Z","job_size_bytes":467,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-cc9e8661d9a57e9d1bbe8c4b: done: 0.923027 sec","job_status":"done","queue_duration_s":1.531828,"scheduling_latency_s":1.531854,"gitaly_calls":7,"gitaly_duration_s":0.182989,"redis_calls":51,"redis_duration_s":0.013804,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001173,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005798,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003992,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.079,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.075,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.619629,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000417092,"duration_s":0.923027,"completed_at":"2026-10-02T16:21:25.549Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.107668,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.365254] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.552Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"cf89f362-938d-48f4-a440-3eb79c0590de\"}"],"class":"WebHookWorker","jid":"3fabc163d931188aca71ff8e","created_at":"2026-10-02T16:21:23.186Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7c68ff79e0d5b9f25caf6b0711b53ae278740822b6f135504dc9f8760c2697f5","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.196Z","job_size_bytes":1751,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3fabc163d931188aca71ff8e: start","job_status":"start","queue_duration_s":2.355979,"scheduling_latency_s":2.356014} gitlab # [ 256.533108] nixbot[2696]: INFO: ::ffff:127.0.0.1:56692 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 256.559634] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.759Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"cf89f362-938d-48f4-a440-3eb79c0590de\"}"],"class":"WebHookWorker","jid":"3fabc163d931188aca71ff8e","created_at":"2026-10-02T16:21:23.186Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-448156087d2f4b65","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7c68ff79e0d5b9f25caf6b0711b53ae278740822b6f135504dc9f8760c2697f5","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.196Z","job_size_bytes":1751,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3fabc163d931188aca71ff8e: done: 0.207193 sec","job_status":"done","queue_duration_s":2.355979,"scheduling_latency_s":2.356014,"redis_calls":19,"redis_duration_s":0.005122,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001321,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000492,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002321,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06021435700000666,"cpu_s":0.082053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000541689,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.207193,"completed_at":"2026-10-02T16:21:25.759Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003009,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.569416] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.766Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"52b0bb0fcc26528d70d9e4cd","created_at":"2026-10-02T16:21:23.684Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.687Z","job_size_bytes":23,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-52b0bb0fcc26528d70d9e4cd: start","job_status":"start","queue_duration_s":2.078755,"scheduling_latency_s":2.078789} gitlab # [ 256.639135] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.830Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"52b0bb0fcc26528d70d9e4cd","created_at":"2026-10-02T16:21:23.684Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.687Z","job_size_bytes":23,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-52b0bb0fcc26528d70d9e4cd: done: 0.063775 sec","job_status":"done","queue_duration_s":2.078755,"scheduling_latency_s":2.078789,"redis_calls":10,"redis_duration_s":0.001953,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000221816,"duration_s":0.063775,"completed_at":"2026-10-02T16:21:25.830Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011151,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.648281] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.831Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"baaebc40dd187a6a3bb1c8e8","created_at":"2026-10-02T16:21:23.865Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.869Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-baaebc40dd187a6a3bb1c8e8: start","job_status":"start","queue_duration_s":1.962106,"scheduling_latency_s":1.962121} gitlab # [ 256.688854] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.855Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"baaebc40dd187a6a3bb1c8e8","created_at":"2026-10-02T16:21:23.865Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.869Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-baaebc40dd187a6a3bb1c8e8: done: 0.023746 sec","job_status":"done","queue_duration_s":1.962106,"scheduling_latency_s":1.962121,"redis_calls":10,"redis_duration_s":0.002163,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000332165,"duration_s":0.023746,"completed_at":"2026-10-02T16:21:25.855Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.703759] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.856Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2e6873bd95f887b0be3a9482","created_at":"2026-10-02T16:21:23.880Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.885Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2e6873bd95f887b0be3a9482: start","job_status":"start","queue_duration_s":1.970867,"scheduling_latency_s":1.97088} gitlab # [ 256.718637] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.882Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2e6873bd95f887b0be3a9482","created_at":"2026-10-02T16:21:23.880Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.885Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2e6873bd95f887b0be3a9482: done: 0.025648 sec","job_status":"done","queue_duration_s":1.970867,"scheduling_latency_s":1.97088,"redis_calls":10,"redis_duration_s":0.002595,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000823,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000327416,"duration_s":0.025648,"completed_at":"2026-10-02T16:21:25.882Z","load_balancing_strategy":"primary","db_duration_s":0.006025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.768814] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.883Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bd7d175e702f3c94116ca923","created_at":"2026-10-02T16:21:23.897Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.901Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bd7d175e702f3c94116ca923: start","job_status":"start","queue_duration_s":1.982123,"scheduling_latency_s":1.982137} gitlab # [ 256.771084] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.910Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bd7d175e702f3c94116ca923","created_at":"2026-10-02T16:21:23.897Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.901Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bd7d175e702f3c94116ca923: done: 0.027555 sec","job_status":"done","queue_duration_s":1.982123,"scheduling_latency_s":1.982137,"redis_calls":13,"redis_duration_s":0.002862,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000676,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000283276,"duration_s":0.027555,"completed_at":"2026-10-02T16:21:25.910Z","load_balancing_strategy":"primary","db_duration_s":0.006091,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.820392] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.912Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6a297dc9aa5de4de9649e958","created_at":"2026-10-02T16:21:23.901Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.904Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6a297dc9aa5de4de9649e958: start","job_status":"start","queue_duration_s":2.007665,"scheduling_latency_s":2.007678} gitlab # [ 256.825950] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.935Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6a297dc9aa5de4de9649e958","created_at":"2026-10-02T16:21:23.901Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.904Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6a297dc9aa5de4de9649e958: done: 0.023639 sec","job_status":"done","queue_duration_s":2.007665,"scheduling_latency_s":2.007678,"redis_calls":10,"redis_duration_s":0.002173,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000421,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000325181,"duration_s":0.023639,"completed_at":"2026-10-02T16:21:25.935Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005114,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.873777] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.936Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"23cd2a4685c1f707eaef74e9","created_at":"2026-10-02T16:21:23.904Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.908Z","job_size_bytes":19,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-23cd2a4685c1f707eaef74e9: start","job_status":"start","queue_duration_s":2.028793,"scheduling_latency_s":2.028807} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45' gitlab # [ 256.879785] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.959Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"23cd2a4685c1f707eaef74e9","created_at":"2026-10-02T16:21:23.904Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.908Z","job_size_bytes":19,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-23cd2a4685c1f707eaef74e9: done: 0.022136 sec","job_status":"done","queue_duration_s":2.028793,"scheduling_latency_s":2.028807,"redis_calls":11,"redis_duration_s":0.002431,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001514,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345016,"duration_s":0.022136,"completed_at":"2026-10-02T16:21:25.959Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004779,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.927399] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:25.960Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3ecf887fbb45f14b1eadeac7","created_at":"2026-10-02T16:21:23.910Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.912Z","job_size_bytes":66,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3ecf887fbb45f14b1eadeac7: start","job_status":"start","queue_duration_s":2.047798,"scheduling_latency_s":2.047821} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45', in 0.04 seconds) gitlab # [ 256.934550] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.055Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3ecf887fbb45f14b1eadeac7","created_at":"2026-10-02T16:21:23.910Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.912Z","job_size_bytes":66,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3ecf887fbb45f14b1eadeac7: done: 0.094475 sec","job_status":"done","queue_duration_s":2.047798,"scheduling_latency_s":2.047821,"redis_calls":10,"redis_duration_s":0.002049,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.073,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.07,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000267073,"duration_s":0.094475,"completed_at":"2026-10-02T16:21:26.055Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.073296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.999618] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4927cf9c047b073722ed70cc","created_at":"2026-10-02T16:21:23.913Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.916Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4927cf9c047b073722ed70cc: start","job_status":"start","queue_duration_s":2.140229,"scheduling_latency_s":2.140244} gitlab # [ 257.000469] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4927cf9c047b073722ed70cc","created_at":"2026-10-02T16:21:23.913Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.916Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4927cf9c047b073722ed70cc: done: 0.026505 sec","job_status":"done","queue_duration_s":2.140229,"scheduling_latency_s":2.140244,"redis_calls":10,"redis_duration_s":0.001809,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000337,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000231873,"duration_s":0.026505,"completed_at":"2026-10-02T16:21:26.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.049128] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.083Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1e21bf77f5a3cc681e953ac4","created_at":"2026-10-02T16:21:23.920Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.923Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1e21bf77f5a3cc681e953ac4: start","job_status":"start","queue_duration_s":2.160377,"scheduling_latency_s":2.160393} gitlab # [ 257.050150] nixbot[2696]: INFO: ::ffff:127.0.0.1:56700 - "GET /api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 200 OK gitlab # [ 257.050919] gitaly[919]: time="2026-10-02T16:21:26.171Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=146 command.spawn_token_fork_ms=144 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ9TZW0AS6QW36A8GKQM2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:26.023" grpc.time_ms=148.027 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.056334] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 188 gitlab # [ 257.057964] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.106Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1e21bf77f5a3cc681e953ac4","created_at":"2026-10-02T16:21:23.920Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.923Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1e21bf77f5a3cc681e953ac4: done: 0.022961 sec","job_status":"done","queue_duration_s":2.160377,"scheduling_latency_s":2.160393,"redis_calls":10,"redis_duration_s":0.001284,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203936,"duration_s":0.022961,"completed_at":"2026-10-02T16:21:26.106Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005955,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.114929] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6f6d0dab2ca5fad2712eea74","created_at":"2026-10-02T16:21:23.924Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.926Z","job_size_bytes":6,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6f6d0dab2ca5fad2712eea74: start","job_status":"start","queue_duration_s":2.181285,"scheduling_latency_s":2.181299} gitlab # [ 257.115807] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6f6d0dab2ca5fad2712eea74","created_at":"2026-10-02T16:21:23.924Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.926Z","job_size_bytes":6,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6f6d0dab2ca5fad2712eea74: done: 0.014241 sec","job_status":"done","queue_duration_s":2.181285,"scheduling_latency_s":2.181299,"redis_calls":10,"redis_duration_s":0.001287,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000255619,"duration_s":0.014241,"completed_at":"2026-10-02T16:21:26.122Z","load_balancing_strategy":"primary","db_duration_s":0.001338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.154262] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.123Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6297753c382aad970be881d3","created_at":"2026-10-02T16:21:23.927Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.929Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6297753c382aad970be881d3: start","job_status":"start","queue_duration_s":2.193399,"scheduling_latency_s":2.193412} gitlab # [ 257.200705] gitaly[919]: time="2026-10-02T16:21:26.312Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 diskcache=352e8a59-ba56-4a87-85ae-871866d13cf5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:21:26.299" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:26.300" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.201518] gitaly[919]: time="2026-10-02T16:21:26.313Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=311 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:21:26.299" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:26.300" grpc.time_ms=12.762 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.202271] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.218Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6297753c382aad970be881d3","created_at":"2026-10-02T16:21:23.927Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.929Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6297753c382aad970be881d3: done: 0.095778 sec","job_status":"done","queue_duration_s":2.193399,"scheduling_latency_s":2.193412,"redis_calls":10,"redis_duration_s":0.001848,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000504,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000390552,"duration_s":0.095778,"completed_at":"2026-10-02T16:21:26.218Z","load_balancing_strategy":"primary","db_duration_s":0.007056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.257363] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.220Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"51929d308b89f5ca620b5308","created_at":"2026-10-02T16:21:23.934Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.937Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-51929d308b89f5ca620b5308: start","job_status":"start","queue_duration_s":2.282491,"scheduling_latency_s":2.282508} gitlab # [ 257.258126] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.266Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"51929d308b89f5ca620b5308","created_at":"2026-10-02T16:21:23.934Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:23.937Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-51929d308b89f5ca620b5308: done: 0.046717 sec","job_status":"done","queue_duration_s":2.282491,"scheduling_latency_s":2.282508,"redis_calls":14,"redis_duration_s":0.0028130000000000004,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000257296,"duration_s":0.046717,"completed_at":"2026-10-02T16:21:26.266Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012569,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.282592] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.268Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7209a18da01b7c7f92e1abc0","created_at":"2026-10-02T16:21:23.998Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.001Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7209a18da01b7c7f92e1abc0: start","job_status":"start","queue_duration_s":2.266464,"scheduling_latency_s":2.266478} gitlab # [ 257.329963] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.286Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7209a18da01b7c7f92e1abc0","created_at":"2026-10-02T16:21:23.998Z","trace_propagation_headers":{"sentry-trace":"d3f8374a6e964ecf844f0ac6c29c4dea-a3c600971c5f4210","baggage":"sentry-trace_id=d3f8374a6e964ecf844f0ac6c29c4dea,sentry-release=Unknown"},"correlation_id":"01M3YPNX067G5VTC6YF1RJGJGW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.001Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7209a18da01b7c7f92e1abc0: done: 0.018524 sec","job_status":"done","queue_duration_s":2.266464,"scheduling_latency_s":2.266478,"redis_calls":8,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000337,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018524,"completed_at":"2026-10-02T16:21:26.286Z","load_balancing_strategy":"primary","db_duration_s":0.003355,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 257.370288] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.287Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"80549d9a03ae351bf62da11b","created_at":"2026-10-02T16:21:24.138Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-3f82ebd9562349fc","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"5add38727fe95c6d6eaee018","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.140Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-80549d9a03ae351bf62da11b: start","job_status":"start","queue_duration_s":2.147361,"scheduling_latency_s":2.147375} gitlab # [ 257.371027] gitaly[919]: time="2026-10-02T16:21:26.479Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQA0YY2BQ1YE7SD59H3WR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:26.471" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 257.371587] gitlab-workhorse[828]: time="2026-10-02T16:21:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 257.372084] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 305 gitlab # [ 257.372574] gitlab-workhorse[828]: time="2026-10-02T16:21:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=336 gitlab # [ 257.373097] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 49 gitlab # [ 257.373570] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.328Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"80549d9a03ae351bf62da11b","created_at":"2026-10-02T16:21:24.138Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-3f82ebd9562349fc","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"5add38727fe95c6d6eaee018","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.140Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-80549d9a03ae351bf62da11b: done: 0.041026 sec","job_status":"done","queue_duration_s":2.147361,"scheduling_latency_s":2.147375,"gitaly_calls":1,"gitaly_duration_s":0.015035,"redis_calls":12,"redis_duration_s":0.0024289999999999997,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000492,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.027508245,"duration_s":0.041026,"completed_at":"2026-10-02T16:21:26.328Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.416639] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:26.330Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"52e2a08385058ace9157be14","created_at":"2026-10-02T16:21:24.243Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-3f82ebd9562349fc","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bac8a037ea3d2b9d2271449276a248a642919b282c699d9775fc2771dfbe84f4","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.245Z","job_size_bytes":467,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-52e2a08385058ace9157be14: start","job_status":"start","queue_duration_s":2.084488,"scheduling_latency_s":2.084504} gitlab # [ 257.421274] gitaly[919]: time="2026-10-02T16:21:26.479Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQA0YY2BQ1YE7SD59H3WR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:26.471" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.467699] gitaly[919]: time="2026-10-02T16:21:26.479Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQA0YY2BQ1YE7SD59H3WR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:26.471" grpc.time_ms=8.259 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.468556] gitaly[919]: time="2026-10-02T16:21:26.530Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAAJE1XZRY59D2YXV4C9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:26.519" pid=919 remote_ip="2001:db8:1::1" request_sha=655689795527ad20a792192dd4d12fdf428eaa05 response_bytes=336 span.kind=server system=grpc gitlab # [ 257.469214] gitaly[919]: time="2026-10-02T16:21:26.530Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1190 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAAJE1XZRY59D2YXV4C9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:26.519" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.469816] gitaly[919]: time="2026-10-02T16:21:26.530Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1190 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAAJE1XZRY59D2YXV4C9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:26.519" grpc.time_ms=11.48 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.520775] gitaly[919]: time="2026-10-02T16:21:26.721Z" level=info msg="finished unary call with code OK" catfile.duration_ms=372 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=372 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:56.247" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:26.348" grpc.time_ms=373.192 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.531272] gitaly[919]: time="2026-10-02T16:21:26.731Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:21:36.005" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:26.725" grpc.time_ms=6.36 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.558145] gitaly[919]: time="2026-10-02T16:21:26.758Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:21:56.657" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:26.757" grpc.time_ms=0.854 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.562303] nixbot[2696]: {"ts": "2026-10-02T16:21:26.762Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 257.563154] gitaly[919]: time="2026-10-02T16:21:26.763Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=143 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T16:21:56.661" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:26.761" grpc.time_ms=1.853 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.616089] gitaly[919]: time="2026-10-02T16:21:26.816Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:36.090" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:26.811" grpc.time_ms=5.552 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.629511] gitaly[919]: time="2026-10-02T16:21:26.830Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:56.628" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:26.829" grpc.time_ms=0.653 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.673960] gitaly[919]: time="2026-10-02T16:21:26.874Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:36.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:26.868" grpc.time_ms=6.334 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.712803] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 136 gitlab # [ 257.714967] nixbot[2696]: {"ts": "2026-10-02T16:21:26.915Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c \"HTTP/1.1 201 Created\""} gitlab # [ 257.766765] gitaly[919]: time="2026-10-02T16:21:26.967Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAR6G6WXW797MJF6AWND grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:56.866" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:26.966" grpc.time_ms=0.648 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.809694] gitaly[919]: time="2026-10-02T16:21:27.010Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQAR6G6WXW797MJF6AWND grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:37.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:27.003" grpc.time_ms=6.351 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.882171] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 164 gitlab # [ 257.889264] nixbot[2696]: {"ts": "2026-10-02T16:21:27.089Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c \"HTTP/1.1 201 Created\""} gitlab # [ 257.952855] gitaly[919]: time="2026-10-02T16:21:27.153Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=957 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:21:57.040" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:27.140" grpc.time_ms=12.684 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45' gitlab # [ 258.004308] gitaly[919]: time="2026-10-02T16:21:27.204Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQ56T2BVNDJ8BVS72WEE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T16:21:57.088" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:27.188" grpc.time_ms=15.848 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.017852] sidekiq-cluster[2573]: {"severity":"WARN","time":"2026-10-02T16:21:27.217Z","class":"CreatePipelineWorker","correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"80d6afdddfb9366d3eeba0eaf297ab097f89953c","after":"763551d41e8de88b31a5ee89e2050c071ec59b45","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"763551d41e8de88b31a5ee89e2050c071ec59b45","ref":"refs/heads/master"},"retry":0} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45', in 0.04 seconds) gitlab # [ 258.026759] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"52e2a08385058ace9157be14","created_at":"2026-10-02T16:21:24.243Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-3f82ebd9562349fc","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bac8a037ea3d2b9d2271449276a248a642919b282c699d9775fc2771dfbe84f4","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.245Z","job_size_bytes":467,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-52e2a08385058ace9157be14: done: 0.89566 sec","job_status":"done","queue_duration_s":2.084488,"scheduling_latency_s":2.084504,"gitaly_calls":7,"gitaly_duration_s":0.502213,"redis_calls":33,"redis_duration_s":0.020612000000000002,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000555,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.013051,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004556,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.35726,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.89566,"completed_at":"2026-10-02T16:21:27.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015252,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 258.036286] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.233Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"b19dafee-c342-49e7-87be-cbf6996fe454\"}"],"class":"WebHookWorker","jid":"64be063187e0749215323640","created_at":"2026-10-02T16:21:24.255Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-3f82ebd9562349fc","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fcd684f5dedf1e3b527d56307c50f950d839435d6ffbb9276ac7db58543f51b0","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.257Z","job_size_bytes":1749,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-64be063187e0749215323640: start","job_status":"start","queue_duration_s":2.976154,"scheduling_latency_s":2.976194} gitlab # [ 258.054555] gitaly[919]: time="2026-10-02T16:21:27.236Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAXW95CJN3MM5466MDWZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:57.134" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:27.234" grpc.time_ms=1.212 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.066324] nixbot[2696]: INFO: ::ffff:127.0.0.1:56714 - "GET /api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 200 OK gitlab # [ 258.100989] nixbot[2696]: INFO: ::ffff:127.0.0.1:56716 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 258.114866] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.315Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"b19dafee-c342-49e7-87be-cbf6996fe454\"}"],"class":"WebHookWorker","jid":"64be063187e0749215323640","created_at":"2026-10-02T16:21:24.255Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-3f82ebd9562349fc","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fcd684f5dedf1e3b527d56307c50f950d839435d6ffbb9276ac7db58543f51b0","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.257Z","job_size_bytes":1749,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-64be063187e0749215323640: done: 0.082036 sec","job_status":"done","queue_duration_s":2.976154,"scheduling_latency_s":2.976194,"redis_calls":14,"redis_duration_s":0.011018,"redis_read_bytes":25,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.002138,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00583,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00305,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0356738079999559,"cpu_s":0.021018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.082036,"completed_at":"2026-10-02T16:21:27.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.157489] gitaly[919]: time="2026-10-02T16:21:27.358Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=44 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQAXW95CJN3MM5466MDWZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:37.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:27.311" grpc.time_ms=46.743 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.172076] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.371Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"931cd94bbc1d46579763a385","created_at":"2026-10-02T16:21:24.370Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-c33fd457bcbe4796","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.374Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-931cd94bbc1d46579763a385: start","job_status":"start","queue_duration_s":2.997348,"scheduling_latency_s":2.997383} gitlab # [ 258.231750] gitaly[919]: time="2026-10-02T16:21:27.432Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQB66T91ZS79X6BHBRZ09 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.422" grpc.time_ms=9.872 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.232954] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:27 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 66 gitlab # [ 258.241621] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.441Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"931cd94bbc1d46579763a385","created_at":"2026-10-02T16:21:24.370Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-c33fd457bcbe4796","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:24.374Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-931cd94bbc1d46579763a385: done: 0.070467 sec","job_status":"done","queue_duration_s":2.997348,"scheduling_latency_s":2.997383,"redis_calls":19,"redis_duration_s":0.006855999999999999,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001638,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000448,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.004115,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000655,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.013027913,"duration_s":0.070467,"completed_at":"2026-10-02T16:21:27.441Z","load_balancing_strategy":"primary","db_duration_s":0.007404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.283959] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.443Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3fabc163d931188aca71ff8e"],"class":"WebHooks::LogExecutionWorker","jid":"4dc7b3c52ee818bdd5652080","created_at":"2026-10-02T16:21:25.737Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-8fcc1f37a45e4d87","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8939425aae98f1fadfcc0a49ee082abf08c3250ccb53816cda2c647efbd18a43","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:25.748Z","job_size_bytes":2593,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4dc7b3c52ee818bdd5652080: start","job_status":"start","queue_duration_s":1.694806,"scheduling_latency_s":1.694824} gitlab # [ 258.284720] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.480Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3fabc163d931188aca71ff8e"],"class":"WebHooks::LogExecutionWorker","jid":"4dc7b3c52ee818bdd5652080","created_at":"2026-10-02T16:21:25.737Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-8fcc1f37a45e4d87","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8939425aae98f1fadfcc0a49ee082abf08c3250ccb53816cda2c647efbd18a43","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:25.748Z","job_size_bytes":2593,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4dc7b3c52ee818bdd5652080: done: 0.037373 sec","job_status":"done","queue_duration_s":1.694806,"scheduling_latency_s":1.694824,"redis_calls":10,"redis_duration_s":0.001593,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000305,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000326,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239416,"duration_s":0.037373,"completed_at":"2026-10-02T16:21:27.480Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006207,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.303327] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.482Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"cd13dc6562f95881f24c69bf","created_at":"2026-10-02T16:21:26.900Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-629f65c227a6470b","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:26.902Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-cd13dc6562f95881f24c69bf: start","job_status":"start","queue_duration_s":0.579388,"scheduling_latency_s":0.579404} gitlab # [ 258.358463] gitlab-workhorse[828]: time="2026-10-02T16:21:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 258.360447] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 54 gitlab # [ 258.362450] gitaly[919]: time="2026-10-02T16:21:27.487Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQB8A3A9DM62SWGKVZJR6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.478" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 258.372241] gitaly[919]: time="2026-10-02T16:21:27.487Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQB8A3A9DM62SWGKVZJR6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.478" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.384680] nixbot[2696]: {"ts": "2026-10-02T16:21:27.567Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c \"HTTP/1.1 201 Created\""} gitlab # [ 258.387414] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/80d6afdddfb9366d3eeba0eaf297ab097f89953c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 464 gitlab # [ 258.389795] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:27 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 53 gitlab # [ 258.391821] gitaly[919]: time="2026-10-02T16:21:27.488Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQB8A3A9DM62SWGKVZJR6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.478" grpc.time_ms=9.299 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.405489] gitaly[919]: time="2026-10-02T16:21:27.567Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQBATAZBRKPKNCZXY802R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.558" grpc.time_ms=8.881 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.420623] gitaly[919]: time="2026-10-02T16:21:27.621Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:27.620" grpc.time_ms=0.342 pid=919 span.kind=server system=grpc gitlab # [ 258.438341] gitaly[919]: time="2026-10-02T16:21:27.638Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBCHW9K8PS262R5EKPY9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.630" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 258.439121] gitaly[919]: time="2026-10-02T16:21:27.639Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBCHW9K8PS262R5EKPY9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.630" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.440270] gitaly[919]: time="2026-10-02T16:21:27.640Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBCHW9K8PS262R5EKPY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.630" grpc.time_ms=8.248 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.444818] gitlab-workhorse[828]: time="2026-10-02T16:21:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 258.447599] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 72 gitlab # [ 258.485397] gitaly[919]: time="2026-10-02T16:21:27.686Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:27.685" grpc.time_ms=0.54 pid=919 span.kind=server system=grpc gitlab # [ 258.514894] gitaly[919]: time="2026-10-02T16:21:27.715Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:27.715" grpc.time_ms=0.243 pid=919 span.kind=server system=grpc gitlab # [ 258.524095] gitaly[919]: time="2026-10-02T16:21:27.722Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQBFNC4CA7MANXF696NYG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.715" grpc.time_ms=6.758 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.524833] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:27 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 53 gitlab # [ 258.533716] gitaly[919]: time="2026-10-02T16:21:27.734Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD diskcache=0e8bd0c4-ebf2-42cf-ad6f-0ac54443000f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:21:57.499" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:27.600" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.536405] gitaly[919]: time="2026-10-02T16:21:27.735Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=111 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7093 command.oublock=0 command.real_time_ms=126 command.spawn_token_fork_ms=0 command.system_time_ms=85 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:21:57.499" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:27.600" grpc.time_ms=135.25 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.576098] gitaly[919]: time="2026-10-02T16:21:27.776Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBHCWPA89VYRC5BSYAC3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.764" pid=919 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 258.576688] gitaly[919]: time="2026-10-02T16:21:27.776Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBHCWPA89VYRC5BSYAC3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.764" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.577501] gitlab-workhorse[828]: time="2026-10-02T16:21:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 258.577957] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 52 gitlab # [ 258.578603] gitaly[919]: time="2026-10-02T16:21:27.776Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBHCWPA89VYRC5BSYAC3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:21:27.764" grpc.time_ms=11.95 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 258.615266] gitaly[919]: time="2026-10-02T16:21:27.815Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=29 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=29 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:21:57.684" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:27.785" grpc.time_ms=30.477 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.643669] gitaly[919]: time="2026-10-02T16:21:27.844Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQAKR9DDZ0R718K6C7GXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:57.743" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:27.843" grpc.time_ms=0.495 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.656405] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.850Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"cd13dc6562f95881f24c69bf","created_at":"2026-10-02T16:21:26.900Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-629f65c227a6470b","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:26.902Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2573,"job_status":"rescheduled","message":"PipelineProcessWorker JID-cd13dc6562f95881f24c69bf: rescheduled"} gitlab # [ 258.656967] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.855Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"cd13dc6562f95881f24c69bf","created_at":"2026-10-02T16:21:26.900Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-629f65c227a6470b","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:26.902Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-cd13dc6562f95881f24c69bf: done: 0.373137 sec","job_status":"done","queue_duration_s":0.579388,"scheduling_latency_s":0.579404,"gitaly_calls":3,"gitaly_duration_s":0.172871,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.010229,"redis_read_bytes":148,"redis_write_bytes":18017,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000455,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000719,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002596,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12316,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.003902,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000341,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000399,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001362,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.046,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.046,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10271,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.338058328,"duration_s":0.373137,"completed_at":"2026-10-02T16:21:27.855Z","load_balancing_strategy":"primary","db_duration_s":0.076859,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 258.706736] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.856Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","64be063187e0749215323640"],"class":"WebHooks::LogExecutionWorker","jid":"48bc42c561fcd6ff43fb2e8f","created_at":"2026-10-02T16:21:27.304Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-51c3c99eba584b25","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e96eb5f9624edc354b92b99bdbf9f19154d326dc3499c5e8f947bfab6ef3d235","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.306Z","job_size_bytes":2591,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-48bc42c561fcd6ff43fb2e8f: start","job_status":"start","queue_duration_s":0.55032,"scheduling_latency_s":0.550337} gitlab # [ 258.707501] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.880Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","64be063187e0749215323640"],"class":"WebHooks::LogExecutionWorker","jid":"48bc42c561fcd6ff43fb2e8f","created_at":"2026-10-02T16:21:27.304Z","trace_propagation_headers":{"sentry-trace":"f04bb59ec80345ddaaa17793a892ae1e-51c3c99eba584b25","baggage":"sentry-trace_id=f04bb59ec80345ddaaa17793a892ae1e,sentry-release=Unknown"},"correlation_id":"01M3YPQ56T2BVNDJ8BVS72WEE8","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e96eb5f9624edc354b92b99bdbf9f19154d326dc3499c5e8f947bfab6ef3d235","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.306Z","job_size_bytes":2591,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-48bc42c561fcd6ff43fb2e8f: done: 0.023966 sec","job_status":"done","queue_duration_s":0.55032,"scheduling_latency_s":0.550337,"redis_calls":7,"redis_duration_s":0.001391,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023966,"completed_at":"2026-10-02T16:21:27.880Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.767372] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.881Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"25d3248187d5bbd8e801c577","created_at":"2026-10-02T16:21:27.420Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-e5ed99f5becc4f81","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.427Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-25d3248187d5bbd8e801c577: start","job_status":"start","queue_duration_s":0.453894,"scheduling_latency_s":0.45394} gitlab # [ 258.767994] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.946Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"25d3248187d5bbd8e801c577","created_at":"2026-10-02T16:21:27.420Z","trace_propagation_headers":{"sentry-trace":"a7a0a7cb24854abb8244e9de1152335f-e5ed99f5becc4f81","baggage":"sentry-trace_id=a7a0a7cb24854abb8244e9de1152335f,sentry-release=Unknown"},"correlation_id":"01M3YPNQW9VV9PQ86T6618P9AW","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.427Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-25d3248187d5bbd8e801c577: done: 0.064774 sec","job_status":"done","queue_duration_s":0.453894,"scheduling_latency_s":0.45394,"redis_calls":11,"redis_duration_s":0.001723,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001153,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000220698,"duration_s":0.064774,"completed_at":"2026-10-02T16:21:27.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015569,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.818624] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"350ccc3ab660ad018456e28e","created_at":"2026-10-02T16:21:27.594Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.595Z","job_size_bytes":23,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-350ccc3ab660ad018456e28e: start","job_status":"start","queue_duration_s":0.351852,"scheduling_latency_s":0.351868} gitlab # [ 258.819556] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.976Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"350ccc3ab660ad018456e28e","created_at":"2026-10-02T16:21:27.594Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.595Z","job_size_bytes":23,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-350ccc3ab660ad018456e28e: done: 0.028855 sec","job_status":"done","queue_duration_s":0.351852,"scheduling_latency_s":0.351868,"redis_calls":7,"redis_duration_s":0.001553,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028855,"completed_at":"2026-10-02T16:21:27.976Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005338,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 258.863178] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.977Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6aa3a8a5a6799a00f30508ae","created_at":"2026-10-02T16:21:27.740Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.742Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6aa3a8a5a6799a00f30508ae: start","job_status":"start","queue_duration_s":0.235369,"scheduling_latency_s":0.235393} gitlab # [ 258.864165] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.997Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6aa3a8a5a6799a00f30508ae","created_at":"2026-10-02T16:21:27.740Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.742Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6aa3a8a5a6799a00f30508ae: done: 0.019333 sec","job_status":"done","queue_duration_s":0.235369,"scheduling_latency_s":0.235393,"redis_calls":7,"redis_duration_s":0.0010040000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019333,"completed_at":"2026-10-02T16:21:27.997Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.934957] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:27.998Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"44146621aa0a54c170466f07","created_at":"2026-10-02T16:21:27.749Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.752Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-44146621aa0a54c170466f07: start","job_status":"start","queue_duration_s":0.24572,"scheduling_latency_s":0.245735} gitlab # [ 258.936104] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.014Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"44146621aa0a54c170466f07","created_at":"2026-10-02T16:21:27.749Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.752Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-44146621aa0a54c170466f07: done: 0.016927 sec","job_status":"done","queue_duration_s":0.24572,"scheduling_latency_s":0.245735,"redis_calls":7,"redis_duration_s":0.001146,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016927,"completed_at":"2026-10-02T16:21:28.014Z","load_balancing_strategy":"primary","db_duration_s":0.002689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.980611] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.015Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bffec1c23b93dc050509f8e9","created_at":"2026-10-02T16:21:27.758Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.759Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bffec1c23b93dc050509f8e9: start","job_status":"start","queue_duration_s":0.255948,"scheduling_latency_s":0.255963} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45' gitlab # [ 258.981703] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.035Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bffec1c23b93dc050509f8e9","created_at":"2026-10-02T16:21:27.758Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.759Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bffec1c23b93dc050509f8e9: done: 0.019798 sec","job_status":"done","queue_duration_s":0.255948,"scheduling_latency_s":0.255963,"redis_calls":10,"redis_duration_s":0.001563,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000839,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000416,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019798,"completed_at":"2026-10-02T16:21:28.035Z","load_balancing_strategy":"primary","db_duration_s":0.003883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.007889] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.036Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6c69e65ff49fc8ba3a689907","created_at":"2026-10-02T16:21:27.760Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.761Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6c69e65ff49fc8ba3a689907: start","job_status":"start","queue_duration_s":0.275278,"scheduling_latency_s":0.275301} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45', in 0.05 seconds) (finished: subtest: push delivers the webhook, in 49.95 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 259.055120] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.052Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6c69e65ff49fc8ba3a689907","created_at":"2026-10-02T16:21:27.760Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.761Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6c69e65ff49fc8ba3a689907: done: 0.015598 sec","job_status":"done","queue_duration_s":0.275278,"scheduling_latency_s":0.275301,"redis_calls":7,"redis_duration_s":0.001062,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015598,"completed_at":"2026-10-02T16:21:28.052Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.059775] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.053Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"68fd0145a097cce8764abccb","created_at":"2026-10-02T16:21:27.762Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.764Z","job_size_bytes":19,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-68fd0145a097cce8764abccb: start","job_status":"start","queue_duration_s":0.289084,"scheduling_latency_s":0.289098} gitlab # [ 259.075793] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.068Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"68fd0145a097cce8764abccb","created_at":"2026-10-02T16:21:27.762Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.764Z","job_size_bytes":19,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-68fd0145a097cce8764abccb: done: 0.01495 sec","job_status":"done","queue_duration_s":0.289084,"scheduling_latency_s":0.289098,"redis_calls":8,"redis_duration_s":0.0013419999999999999,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01495,"completed_at":"2026-10-02T16:21:28.068Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.116269] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.070Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d804530337fe542449ef7084","created_at":"2026-10-02T16:21:27.765Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.766Z","job_size_bytes":66,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d804530337fe542449ef7084: start","job_status":"start","queue_duration_s":0.304028,"scheduling_latency_s":0.304071} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.09 seconds) gitlab # [ 259.130330] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.090Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d804530337fe542449ef7084","created_at":"2026-10-02T16:21:27.765Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.766Z","job_size_bytes":66,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d804530337fe542449ef7084: done: 0.020505 sec","job_status":"done","queue_duration_s":0.304028,"scheduling_latency_s":0.304071,"redis_calls":7,"redis_duration_s":0.001161,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020505,"completed_at":"2026-10-02T16:21:28.090Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.199771] nixbot[2696]: INFO: ::ffff:127.0.0.1:56732 - "GET /api/repos/gitlab/root/test-flake/builds?commit=763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 200 OK gitlab # [ 259.200641] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 67 gitlab # [ 259.201387] gitaly[919]: time="2026-10-02T16:21:28.359Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQC3JFZSEMZKG5855GDZ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:58.057" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:28.358" grpc.time_ms=0.522 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.202083] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.091Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ae14aa06cb9d96cd280c9c70","created_at":"2026-10-02T16:21:27.766Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.767Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ae14aa06cb9d96cd280c9c70: start","job_status":"start","queue_duration_s":0.324155,"scheduling_latency_s":0.324172} gitlab # [ 259.202701] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.113Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ae14aa06cb9d96cd280c9c70","created_at":"2026-10-02T16:21:27.766Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.767Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ae14aa06cb9d96cd280c9c70: done: 0.022111 sec","job_status":"done","queue_duration_s":0.324155,"scheduling_latency_s":0.324172,"redis_calls":7,"redis_duration_s":0.0013579999999999998,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022111,"completed_at":"2026-10-02T16:21:28.113Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.263677] gitaly[919]: time="2026-10-02T16:21:28.387Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:58.186" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:28.386" grpc.time_ms=0.491 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.264682] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.114Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ebf27207f1852ce326d5188f","created_at":"2026-10-02T16:21:27.767Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.768Z","job_size_bytes":6,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ebf27207f1852ce326d5188f: start","job_status":"start","queue_duration_s":0.346199,"scheduling_latency_s":0.346223} gitlab # [ 259.270093] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ebf27207f1852ce326d5188f","created_at":"2026-10-02T16:21:27.767Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.768Z","job_size_bytes":6,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ebf27207f1852ce326d5188f: done: 0.010407 sec","job_status":"done","queue_duration_s":0.346199,"scheduling_latency_s":0.346223,"redis_calls":7,"redis_duration_s":0.001115,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010407,"completed_at":"2026-10-02T16:21:28.125Z","load_balancing_strategy":"primary","db_duration_s":0.000487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.323752] gitaly[919]: time="2026-10-02T16:21:28.402Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T16:21:58.195" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:28.396" grpc.time_ms=6.627 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.324635] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.126Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"086553431e30153ff15d038a","created_at":"2026-10-02T16:21:27.769Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.769Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-086553431e30153ff15d038a: start","job_status":"start","queue_duration_s":0.356282,"scheduling_latency_s":0.356296} gitlab # [ 259.370686] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.141Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"086553431e30153ff15d038a","created_at":"2026-10-02T16:21:27.769Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.769Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-086553431e30153ff15d038a: done: 0.015378 sec","job_status":"done","queue_duration_s":0.356282,"scheduling_latency_s":0.356296,"redis_calls":7,"redis_duration_s":0.0010609999999999999,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000317,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015378,"completed_at":"2026-10-02T16:21:28.141Z","load_balancing_strategy":"primary","db_duration_s":0.002608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.376567] gitaly[919]: time="2026-10-02T16:21:28.413Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:38.267" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:28.408" grpc.time_ms=4.775 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.387602] nixbot[2696]: {"ts": "2026-10-02T16:21:28.574Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/763551d41e8de88b31a5ee89e2050c071ec59b45 \"HTTP/1.1 201 Created\""} gitlab # [ 259.390577] nixbot[2696]: {"ts": "2026-10-02T16:21:28.583Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-763551d41e8d-5"} gitlab # [ 259.392387] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.142Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"9ece9629da1a6c4710d96c6a","created_at":"2026-10-02T16:21:27.850Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.851Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9ece9629da1a6c4710d96c6a: start","job_status":"start","queue_duration_s":0.291014,"scheduling_latency_s":0.29103} gitlab # [ 259.407063] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.159Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"9ece9629da1a6c4710d96c6a","created_at":"2026-10-02T16:21:27.850Z","trace_propagation_headers":{"sentry-trace":"c41a39aff48147c0bc4e9a31fd3e9f9c-0b16c045ef8c4f3a","baggage":"sentry-trace_id=c41a39aff48147c0bc4e9a31fd3e9f9c,sentry-release=Unknown"},"correlation_id":"01M3YPQAKR9DDZ0R718K6C7GXD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:27.851Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9ece9629da1a6c4710d96c6a: done: 0.01689 sec","job_status":"done","queue_duration_s":0.291014,"scheduling_latency_s":0.29103,"redis_calls":8,"redis_duration_s":0.001859,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001368,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01689,"completed_at":"2026-10-02T16:21:28.159Z","load_balancing_strategy":"primary","db_duration_s":0.002404,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 259.456265] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 733 gitlab # [ 259.456862] gitaly[919]: time="2026-10-02T16:21:28.428Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:21:38.288" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:28.428" grpc.time_ms=0.219 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.457876] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.509Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"639ce82cc2be1b08c8d7f0d8","created_at":"2026-10-02T16:21:28.504Z","trace_propagation_headers":{"sentry-trace":"ed3d57fc6f6f4c0e872059ae7b282716-2cbffdaa270d4d01","baggage":"sentry-trace_id=ed3d57fc6f6f4c0e872059ae7b282716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQBMZ5S2CGSRP2D90N5F6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:28.508Z","job_size_bytes":65,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-639ce82cc2be1b08c8d7f0d8: start","job_status":"start","queue_duration_s":0.001388,"scheduling_latency_s":0.001427} gitlab # [ 259.497114] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.536Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"639ce82cc2be1b08c8d7f0d8","created_at":"2026-10-02T16:21:28.504Z","trace_propagation_headers":{"sentry-trace":"ed3d57fc6f6f4c0e872059ae7b282716-2cbffdaa270d4d01","baggage":"sentry-trace_id=ed3d57fc6f6f4c0e872059ae7b282716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQBMZ5S2CGSRP2D90N5F6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:28.508Z","job_size_bytes":65,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-639ce82cc2be1b08c8d7f0d8: done: 0.027151 sec","job_status":"done","queue_duration_s":0.001388,"scheduling_latency_s":0.001427,"redis_calls":7,"redis_duration_s":0.00118,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027151,"completed_at":"2026-10-02T16:21:28.536Z","load_balancing_strategy":"primary","db_duration_s":0.004108,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 259.536704] gitaly[919]: time="2026-10-02T16:21:28.437Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:21:38.291" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:28.431" grpc.time_ms=5.543 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.537636] gitaly[919]: time="2026-10-02T16:21:28.468Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:28.467" grpc.time_ms=0.388 pid=919 span.kind=server system=grpc gitlab # [ 259.538410] nixbot[2696]: {"ts": "2026-10-02T16:21:28.672Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-763551d41e8d-5", "argv0": "bwrap"} gitlab # [ 259.538906] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.537Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"e9ca9eb589975c90e16bd43c","created_at":"2026-10-02T16:21:28.509Z","trace_propagation_headers":{"sentry-trace":"ed3d57fc6f6f4c0e872059ae7b282716-2cbffdaa270d4d01","baggage":"sentry-trace_id=ed3d57fc6f6f4c0e872059ae7b282716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQBMZ5S2CGSRP2D90N5F6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:28.512Z","job_size_bytes":65,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-e9ca9eb589975c90e16bd43c: start","job_status":"start","queue_duration_s":0.025467,"scheduling_latency_s":0.025492} gitlab # [ 259.548946] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.581Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"e9ca9eb589975c90e16bd43c","created_at":"2026-10-02T16:21:28.509Z","trace_propagation_headers":{"sentry-trace":"ed3d57fc6f6f4c0e872059ae7b282716-2cbffdaa270d4d01","baggage":"sentry-trace_id=ed3d57fc6f6f4c0e872059ae7b282716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQBMZ5S2CGSRP2D90N5F6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:28.512Z","job_size_bytes":65,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-e9ca9eb589975c90e16bd43c: done: 0.04359 sec","job_status":"done","queue_duration_s":0.025467,"scheduling_latency_s":0.025492,"redis_calls":10,"redis_duration_s":0.0017660000000000002,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04359,"completed_at":"2026-10-02T16:21:28.581Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.589351] gitaly[919]: time="2026-10-02T16:21:28.490Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:21:28.489" grpc.time_ms=0.495 pid=919 span.kind=server system=grpc gitlab # [ 259.596922] gitaly[919]: time="2026-10-02T16:21:28.499Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 diskcache=0f607a98-ab15-4db6-a1da-c04435500e28 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:21:38.298" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:28.438" pid=919 ref=refs/keep-around/763551d41e8de88b31a5ee89e2050c071ec59b45 remote_ip="2001:db8:1::1" revision=763551d41e8de88b31a5ee89e2050c071ec59b45 span.kind=server system=grpc username=root gitlab # [ 259.635744] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.583Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"416b3c84141dde7445c08f3e","created_at":"2026-10-02T16:21:28.553Z","trace_propagation_headers":{"sentry-trace":"ed3d57fc6f6f4c0e872059ae7b282716-2cbffdaa270d4d01","baggage":"sentry-trace_id=ed3d57fc6f6f4c0e872059ae7b282716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQBMZ5S2CGSRP2D90N5F6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:28.556Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-416b3c84141dde7445c08f3e: start","job_status":"start","queue_duration_s":0.027112,"scheduling_latency_s":0.027147} gitlab # [ 259.636630] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.676Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"416b3c84141dde7445c08f3e","created_at":"2026-10-02T16:21:28.553Z","trace_propagation_headers":{"sentry-trace":"ed3d57fc6f6f4c0e872059ae7b282716-2cbffdaa270d4d01","baggage":"sentry-trace_id=ed3d57fc6f6f4c0e872059ae7b282716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQBMZ5S2CGSRP2D90N5F6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:28.556Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-416b3c84141dde7445c08f3e: done: 0.093105 sec","job_status":"done","queue_duration_s":0.027112,"scheduling_latency_s":0.027147,"gitaly_calls":2,"gitaly_duration_s":0.004707,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.005961,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000774,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000688,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000883,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001676,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000171,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000302,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000693,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059421,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.064411818,"duration_s":0.093105,"completed_at":"2026-10-02T16:21:28.676Z","load_balancing_strategy":"primary","db_duration_s":0.025125,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 259.699190] gitaly[919]: time="2026-10-02T16:21:28.499Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=54 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5565 command.oublock=8 command.real_time_ms=53 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=15 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:21:38.298" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:21:28.438" grpc.time_ms=60.663 pid=919 ref=refs/keep-around/763551d41e8de88b31a5ee89e2050c071ec59b45 remote_ip="2001:db8:1::1" revision=763551d41e8de88b31a5ee89e2050c071ec59b45 span.kind=server system=grpc username=root gitlab # [ 259.700280] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.677Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6aad2ca8d8056deaea5a9a8d","created_at":"2026-10-02T16:21:28.627Z","trace_propagation_headers":{"sentry-trace":"ed3d57fc6f6f4c0e872059ae7b282716-5929ed3424184d6a","baggage":"sentry-trace_id=ed3d57fc6f6f4c0e872059ae7b282716,sentry-release=Unknown"},"correlation_id":"01M3YPQBMZ5S2CGSRP2D90N5F6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:28.629Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6aad2ca8d8056deaea5a9a8d: start","job_status":"start","queue_duration_s":0.048687,"scheduling_latency_s":0.048715} gitlab # [ 259.700876] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:28.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6aad2ca8d8056deaea5a9a8d","created_at":"2026-10-02T16:21:28.627Z","trace_propagation_headers":{"sentry-trace":"ed3d57fc6f6f4c0e872059ae7b282716-5929ed3424184d6a","baggage":"sentry-trace_id=ed3d57fc6f6f4c0e872059ae7b282716,sentry-release=Unknown"},"correlation_id":"01M3YPQBMZ5S2CGSRP2D90N5F6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:28.629Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6aad2ca8d8056deaea5a9a8d: done: 0.040484 sec","job_status":"done","queue_duration_s":0.048687,"scheduling_latency_s":0.048715,"redis_calls":7,"redis_duration_s":0.001144,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040484,"completed_at":"2026-10-02T16:21:28.718Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.755795] gitaly[919]: time="2026-10-02T16:21:28.503Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:58.302" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:28.502" grpc.time_ms=0.353 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.756733] gitaly[919]: time="2026-10-02T16:21:28.640Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:21:58.539" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:28.639" grpc.time_ms=1.062 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.757427] gitaly[919]: time="2026-10-02T16:21:28.664Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQBMZ5S2CGSRP2D90N5F6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:58.563" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:28.663" grpc.time_ms=0.478 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.864579] nix-daemon[2866]: accepted connection from pid 3688, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 260.296663] gitaly[919]: time="2026-10-02T16:21:29.496Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQD6P4YE3SR6G1H1F6D07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:21:59.194" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:29.495" grpc.time_ms=0.971 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.332504] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 102 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.16 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 261.448140] gitaly[919]: time="2026-10-02T16:21:30.647Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQEAHGVKRWDJN0YFXZF1Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:00.346" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:30.646" grpc.time_ms=1.015 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 261.477906] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 100 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.14 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 262.594432] gitaly[919]: time="2026-10-02T16:21:31.794Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQFEC4WV8F41N8H6RAPMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:01.492" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:31.792" grpc.time_ms=1.024 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.633495] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 108 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.16 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 263.754627] gitaly[919]: time="2026-10-02T16:21:32.954Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQGJJ1QKG5A1ZHYCCQG5N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:02.752" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:32.953" grpc.time_ms=1.062 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.794156] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 110 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.16 seconds) gitlab # [ 264.286289] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.485Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"930701788d757bb863c8354e","created_at":"2026-10-02T16:21:33.465Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcee0be38bcfe3c2074538e41e74605d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.481Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-930701788d757bb863c8354e: start","job_status":"start","queue_duration_s":0.003426,"scheduling_latency_s":0.003488} gitlab # [ 264.354109] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.554Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 264.372458] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.572Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"930701788d757bb863c8354e","created_at":"2026-10-02T16:21:33.465Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcee0be38bcfe3c2074538e41e74605d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.481Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-930701788d757bb863c8354e: done: 0.08723 sec","job_status":"done","queue_duration_s":0.003426,"scheduling_latency_s":0.003488,"redis_calls":9,"redis_duration_s":0.009444000000000001,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001815,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003144,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08723,"completed_at":"2026-10-02T16:21:33.572Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.415190] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2e4ae7dda535d6e6948e7cac","created_at":"2026-10-02T16:21:33.521Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"312a8856c6807874ddd30bae827475d3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.538Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2e4ae7dda535d6e6948e7cac: start","job_status":"start","queue_duration_s":0.036481,"scheduling_latency_s":0.036521} gitlab # [ 264.558582] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2e4ae7dda535d6e6948e7cac","created_at":"2026-10-02T16:21:33.521Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"312a8856c6807874ddd30bae827475d3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.538Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2e4ae7dda535d6e6948e7cac: done: 0.183311 sec","job_status":"done","queue_duration_s":0.036481,"scheduling_latency_s":0.036521,"redis_calls":11,"redis_duration_s":0.014539,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002799,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007201,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004539,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001973155,"exclusive_lock_hold_duration_s":0.063106624,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.183311,"completed_at":"2026-10-02T16:21:33.758Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.606770] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"774abc75c1c16a6303f4d80a","created_at":"2026-10-02T16:21:33.552Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3548da86c4be19923b76348e24710070","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:21:33.600Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-774abc75c1c16a6303f4d80a: start","job_status":"start","queue_duration_s":0.163902,"scheduling_latency_s":0.163943} gitlab # [ 264.608244] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.800Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 264.618505] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.818Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"774abc75c1c16a6303f4d80a","created_at":"2026-10-02T16:21:33.552Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3548da86c4be19923b76348e24710070","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:21:33.600Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-774abc75c1c16a6303f4d80a: done: 0.054731 sec","job_status":"done","queue_duration_s":0.163902,"scheduling_latency_s":0.163943,"redis_calls":9,"redis_duration_s":0.011502,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001691,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004425,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005386,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054731,"completed_at":"2026-10-02T16:21:33.818Z","load_balancing_strategy":"primary","db_duration_s":0.006798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.644303] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:33.821Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"eddea01c4d4b7e819705bb99","created_at":"2026-10-02T16:21:33.626Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.635Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-eddea01c4d4b7e819705bb99: start","job_status":"start","queue_duration_s":0.186027,"scheduling_latency_s":0.186068} gitlab # [ 264.755749] nix-daemon[2866]: accepted connection from pid 3725, user nixbot gitlab # [ 264.770341] nix-daemon[2866]: accepted connection from pid 3688, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 264.807218] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.007Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"eddea01c4d4b7e819705bb99","created_at":"2026-10-02T16:21:33.626Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.635Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-eddea01c4d4b7e819705bb99: done: 0.186492 sec","job_status":"done","queue_duration_s":0.186027,"scheduling_latency_s":0.186068,"redis_calls":70,"redis_duration_s":0.04786,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003412,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000983,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.042327,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.186492,"completed_at":"2026-10-02T16:21:34.007Z","load_balancing_strategy":"primary","db_duration_s":0.003212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.849194] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.013Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9749dd64ab6a5a8b9f3e3c16","created_at":"2026-10-02T16:21:33.639Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59be65b0a0b12e170503de8eb421201f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.647Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9749dd64ab6a5a8b9f3e3c16: start","job_status":"start","queue_duration_s":0.366185,"scheduling_latency_s":0.36621} gitlab # [ 264.863876] gitaly[919]: time="2026-10-02T16:21:34.064Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQHNYVXTWEBXW07EESWBH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:04.062" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:34.063" grpc.time_ms=0.79 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.879376] gitaly[919]: time="2026-10-02T16:21:34.079Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQHPGYH5NZ23TR640ZTQS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:04.078" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:34.079" grpc.time_ms=0.456 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.897079] gitaly[919]: time="2026-10-02T16:21:34.095Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQHNYVXTWEBXW07EESWBH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:44.059" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:34.089" grpc.time_ms=6.432 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.908104] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 75 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.11 seconds) gitlab # [ 264.938045] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 123 gitlab # [ 264.939363] nixbot[2696]: {"ts": "2026-10-02T16:21:34.139Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/763551d41e8de88b31a5ee89e2050c071ec59b45 \"HTTP/1.1 201 Created\""} gitlab # [ 264.981465] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.181Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9749dd64ab6a5a8b9f3e3c16","created_at":"2026-10-02T16:21:33.639Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59be65b0a0b12e170503de8eb421201f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.647Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9749dd64ab6a5a8b9f3e3c16: done: 0.168119 sec","job_status":"done","queue_duration_s":0.366185,"scheduling_latency_s":0.36621,"redis_calls":12,"redis_duration_s":0.023229,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.02064,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00232,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.117,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.117,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.121975838,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.168119,"completed_at":"2026-10-02T16:21:34.181Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.116451,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.984100] gitaly[919]: time="2026-10-02T16:21:34.184Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQHSYFP6WEWYBKXPX4VT3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:04.183" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:34.184" grpc.time_ms=0.501 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.038557] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0bab17421311b9a6c3d80fbd","created_at":"2026-10-02T16:21:33.665Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73e43a6c92ebaaa19ffb7e04fc590d0f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.675Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0bab17421311b9a6c3d80fbd: start","job_status":"start","queue_duration_s":0.56306,"scheduling_latency_s":0.563108} gitlab # [ 265.058065] gitaly[919]: time="2026-10-02T16:21:34.258Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQHSYFP6WEWYBKXPX4VT3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:44.218" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:34.248" grpc.time_ms=9.505 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.118661] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 176 gitlab # [ 265.122039] nixbot[2696]: {"ts": "2026-10-02T16:21:34.322Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/763551d41e8de88b31a5ee89e2050c071ec59b45 \"HTTP/1.1 201 Created\""} gitlab # [ 265.130087] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.330Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0bab17421311b9a6c3d80fbd","created_at":"2026-10-02T16:21:33.665Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73e43a6c92ebaaa19ffb7e04fc590d0f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.675Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0bab17421311b9a6c3d80fbd: done: 0.091887 sec","job_status":"done","queue_duration_s":0.56306,"scheduling_latency_s":0.563108,"redis_calls":9,"redis_duration_s":0.0067020000000000005,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001525,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091887,"completed_at":"2026-10-02T16:21:34.330Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012499,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.132049] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.332Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9de16bddcc40edd9388ce919","created_at":"2026-10-02T16:21:33.704Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87c5090e2cc1ddfce89da987d02d7a9e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.715Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9de16bddcc40edd9388ce919: start","job_status":"start","queue_duration_s":0.617625,"scheduling_latency_s":0.617647} gitlab # [ 265.176382] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.364Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9de16bddcc40edd9388ce919","created_at":"2026-10-02T16:21:33.704Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87c5090e2cc1ddfce89da987d02d7a9e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.715Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9de16bddcc40edd9388ce919: done: 0.031667 sec","job_status":"done","queue_duration_s":0.617625,"scheduling_latency_s":0.617647,"redis_calls":11,"redis_duration_s":0.005563,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002336,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000476597,"exclusive_lock_hold_duration_s":0.007562414,"duration_s":0.031667,"completed_at":"2026-10-02T16:21:34.364Z","load_balancing_strategy":"primary","db_duration_s":0.005439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.177974] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"959695a17072d8dcc3ece574","created_at":"2026-10-02T16:21:33.737Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5892699a53a792868d961f74267761ed","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.755Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-959695a17072d8dcc3ece574: start","job_status":"start","queue_duration_s":0.61318,"scheduling_latency_s":0.613197} gitlab # [ 265.290918] gitaly[919]: time="2026-10-02T16:21:34.491Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQJ39XVRNEYB90T6ABCPH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:04.489" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:34.490" grpc.time_ms=0.971 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.302203] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"959695a17072d8dcc3ece574","created_at":"2026-10-02T16:21:33.737Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5892699a53a792868d961f74267761ed","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.755Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-959695a17072d8dcc3ece574: done: 0.133803 sec","job_status":"done","queue_duration_s":0.61318,"scheduling_latency_s":0.613197,"redis_calls":12,"redis_duration_s":0.012712000000000001,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004642,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00379,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.101704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000895365,"exclusive_lock_hold_duration_s":0.09233827300000001,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.133803,"completed_at":"2026-10-02T16:21:34.502Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0096,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} gitlab # [ 265.343786] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ae98c1872fb2cea193bba31c","created_at":"2026-10-02T16:21:33.765Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41035ecb32aa89efa5ed24dcdab71717","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.781Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ae98c1872fb2cea193bba31c: start","job_status":"start","queue_duration_s":0.722745,"scheduling_latency_s":0.722762} gitlab # [ 265.344740] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ae98c1872fb2cea193bba31c","created_at":"2026-10-02T16:21:33.765Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41035ecb32aa89efa5ed24dcdab71717","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.781Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ae98c1872fb2cea193bba31c: done: 0.017745 sec","job_status":"done","queue_duration_s":0.722745,"scheduling_latency_s":0.722762,"redis_calls":11,"redis_duration_s":0.001657,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000294,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00074,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000355073,"exclusive_lock_hold_duration_s":0.002273473,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.017745,"completed_at":"2026-10-02T16:21:34.521Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001989,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.390666] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"57ea2ba9e661e39dbd0488d3","created_at":"2026-10-02T16:21:33.799Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.905Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-57ea2ba9e661e39dbd0488d3: start","job_status":"start","queue_duration_s":0.617723,"scheduling_latency_s":0.617737} gitlab # [ 265.391510] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"57ea2ba9e661e39dbd0488d3","created_at":"2026-10-02T16:21:33.799Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.905Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-57ea2ba9e661e39dbd0488d3: done: 0.025343 sec","job_status":"done","queue_duration_s":0.617723,"scheduling_latency_s":0.617737,"redis_calls":18,"redis_duration_s":0.003067,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000772,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000238,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001546,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000511,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025343,"completed_at":"2026-10-02T16:21:34.548Z","load_balancing_strategy":"primary","db_duration_s":0.002307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.447867] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6390fc5387346c8459703b68","created_at":"2026-10-02T16:21:33.923Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c461bd3769dabfdd34d95e6fd30b77a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:21:33.936Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6390fc5387346c8459703b68: start","job_status":"start","queue_duration_s":0.613392,"scheduling_latency_s":0.613409} gitlab # [ 265.448733] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.558Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 265.449211] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6390fc5387346c8459703b68","created_at":"2026-10-02T16:21:33.923Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c461bd3769dabfdd34d95e6fd30b77a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:21:33.936Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6390fc5387346c8459703b68: done: 0.014428 sec","job_status":"done","queue_duration_s":0.613392,"scheduling_latency_s":0.613409,"redis_calls":9,"redis_duration_s":0.002156,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000546,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014428,"completed_at":"2026-10-02T16:21:34.564Z","load_balancing_strategy":"primary","db_duration_s":0.000615,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.503890] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.565Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b0372ce24061bcc3616741f8","created_at":"2026-10-02T16:21:33.951Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0193526b2a371e526f7658915ce9ca8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:21:33.964Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b0372ce24061bcc3616741f8: start","job_status":"start","queue_duration_s":0.600758,"scheduling_latency_s":0.600773} gitlab # [ 265.504654] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b0372ce24061bcc3616741f8","created_at":"2026-10-02T16:21:33.951Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0193526b2a371e526f7658915ce9ca8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:21:33.964Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b0372ce24061bcc3616741f8: done: 0.023985 sec","job_status":"done","queue_duration_s":0.600758,"scheduling_latency_s":0.600773,"redis_calls":9,"redis_duration_s":0.002283,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023985,"completed_at":"2026-10-02T16:21:34.589Z","load_balancing_strategy":"primary","db_duration_s":0.004549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.515134] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a373cfa28bee02783c91cd30","created_at":"2026-10-02T16:21:33.979Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84077a92926529acb78861dc61c3397f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.989Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a373cfa28bee02783c91cd30: start","job_status":"start","queue_duration_s":0.601329,"scheduling_latency_s":0.601738} gitlab # [ 265.528675] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a373cfa28bee02783c91cd30","created_at":"2026-10-02T16:21:33.979Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84077a92926529acb78861dc61c3397f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.989Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a373cfa28bee02783c91cd30: done: 0.043146 sec","job_status":"done","queue_duration_s":0.601329,"scheduling_latency_s":0.601738,"redis_calls":9,"redis_duration_s":0.002466,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000526,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.043146,"completed_at":"2026-10-02T16:21:34.633Z","load_balancing_strategy":"primary","db_duration_s":0.006743,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.581322] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa5443657928930a27c22983","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa5443657928930a27c22983: start","job_status":"start","queue_duration_s":0.645255,"scheduling_latency_s":0.645284} gitlab # [ 265.582179] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa5443657928930a27c22983","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa5443657928930a27c22983: done: 0.023509 sec","job_status":"done","queue_duration_s":0.645255,"scheduling_latency_s":0.645284,"redis_calls":15,"redis_duration_s":0.002712,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001614,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023509,"completed_at":"2026-10-02T16:21:34.659Z","load_balancing_strategy":"primary","db_duration_s":0.000467,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.617895] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6db5ee20b02ee8fbbd1e0efe","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6db5ee20b02ee8fbbd1e0efe: start","job_status":"start","queue_duration_s":0.669631,"scheduling_latency_s":0.669647} gitlab # [ 265.629808] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6db5ee20b02ee8fbbd1e0efe","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6db5ee20b02ee8fbbd1e0efe: done: 0.010011 sec","job_status":"done","queue_duration_s":0.669631,"scheduling_latency_s":0.669647,"redis_calls":13,"redis_duration_s":0.001693,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010011,"completed_at":"2026-10-02T16:21:34.670Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.674288] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1b7bc248e648999a627a0e0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1b7bc248e648999a627a0e0: start","job_status":"start","queue_duration_s":0.680555,"scheduling_latency_s":0.680568} gitlab # [ 265.677089] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1b7bc248e648999a627a0e0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1b7bc248e648999a627a0e0: done: 0.009726 sec","job_status":"done","queue_duration_s":0.680555,"scheduling_latency_s":0.680568,"redis_calls":13,"redis_duration_s":0.001755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009726,"completed_at":"2026-10-02T16:21:34.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.715034] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2df23f87da11358b1a73089","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2df23f87da11358b1a73089: start","job_status":"start","queue_duration_s":0.691173,"scheduling_latency_s":0.691187} gitlab # [ 265.760074] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2df23f87da11358b1a73089","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2df23f87da11358b1a73089: done: 0.010053 sec","job_status":"done","queue_duration_s":0.691173,"scheduling_latency_s":0.691187,"redis_calls":13,"redis_duration_s":0.0017199999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010053,"completed_at":"2026-10-02T16:21:34.691Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.763091] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c9b7f85871d047aad952c3e","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c9b7f85871d047aad952c3e: start","job_status":"start","queue_duration_s":0.702095,"scheduling_latency_s":0.702109} gitlab # [ 265.775880] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c9b7f85871d047aad952c3e","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c9b7f85871d047aad952c3e: done: 0.0148 sec","job_status":"done","queue_duration_s":0.702095,"scheduling_latency_s":0.702109,"redis_calls":13,"redis_duration_s":0.002325,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001398,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0148,"completed_at":"2026-10-02T16:21:34.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.818781] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9235d599f8e033c86743b474","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9235d599f8e033c86743b474: start","job_status":"start","queue_duration_s":0.718247,"scheduling_latency_s":0.718272} gitlab # [ 265.862244] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9235d599f8e033c86743b474","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9235d599f8e033c86743b474: done: 0.011217 sec","job_status":"done","queue_duration_s":0.718247,"scheduling_latency_s":0.718272,"redis_calls":13,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011217,"completed_at":"2026-10-02T16:21:34.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.863918] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9eaba40c1daba30c85e42959","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9eaba40c1daba30c85e42959: start","job_status":"start","queue_duration_s":0.730399,"scheduling_latency_s":0.730415} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 265.909451] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9eaba40c1daba30c85e42959","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9eaba40c1daba30c85e42959: done: 0.010881 sec","job_status":"done","queue_duration_s":0.730399,"scheduling_latency_s":0.730415,"redis_calls":13,"redis_duration_s":0.001751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010881,"completed_at":"2026-10-02T16:21:34.731Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.910990] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bc34b8c04ef2cb53d3cdcb6","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bc34b8c04ef2cb53d3cdcb6: start","job_status":"start","queue_duration_s":0.742267,"scheduling_latency_s":0.742282} gitlab # [ 265.920945] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bc34b8c04ef2cb53d3cdcb6","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bc34b8c04ef2cb53d3cdcb6: done: 0.012759 sec","job_status":"done","queue_duration_s":0.742267,"scheduling_latency_s":0.742282,"redis_calls":13,"redis_duration_s":0.002225,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001558,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012759,"completed_at":"2026-10-02T16:21:34.745Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.986930] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e98c63700e556a1ecbfe9b7","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e98c63700e556a1ecbfe9b7: start","job_status":"start","queue_duration_s":0.756205,"scheduling_latency_s":0.756225} gitlab # [ 265.987915] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e98c63700e556a1ecbfe9b7","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e98c63700e556a1ecbfe9b7: done: 0.010714 sec","job_status":"done","queue_duration_s":0.756205,"scheduling_latency_s":0.756225,"redis_calls":13,"redis_duration_s":0.0017139999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010714,"completed_at":"2026-10-02T16:21:34.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.043429] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bc55a643442c9a5ee525df0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bc55a643442c9a5ee525df0: start","job_status":"start","queue_duration_s":0.76779,"scheduling_latency_s":0.767805} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.17 seconds) gitlab # [ 266.044069] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bc55a643442c9a5ee525df0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bc55a643442c9a5ee525df0: done: 0.009719 sec","job_status":"done","queue_duration_s":0.76779,"scheduling_latency_s":0.767805,"redis_calls":13,"redis_duration_s":0.001864,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009719,"completed_at":"2026-10-02T16:21:34.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.088727] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7573a2d1be130d72bd6ebcd","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7573a2d1be130d72bd6ebcd: start","job_status":"start","queue_duration_s":0.778421,"scheduling_latency_s":0.778435} gitlab # [ 266.089596] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7573a2d1be130d72bd6ebcd","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7573a2d1be130d72bd6ebcd: done: 0.009843 sec","job_status":"done","queue_duration_s":0.778421,"scheduling_latency_s":0.778435,"redis_calls":13,"redis_duration_s":0.001709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009843,"completed_at":"2026-10-02T16:21:34.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.100455] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c302ca79ca2caf8e0b18ead3","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c302ca79ca2caf8e0b18ead3: start","job_status":"start","queue_duration_s":0.789387,"scheduling_latency_s":0.789401} gitlab # [ 266.112502] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 1313 "" "curl/8.21.0" 115 gitlab # [ 266.114855] gitaly[919]: time="2026-10-02T16:21:35.261Z" level=info msg="finished unary call with code OK" catfile.duration_ms=45 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=45 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQJT0G0SW1C7ZB2P815WG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:05.214" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:35.215" grpc.time_ms=46.064 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.162346] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c302ca79ca2caf8e0b18ead3","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c302ca79ca2caf8e0b18ead3: done: 0.010034 sec","job_status":"done","queue_duration_s":0.789387,"scheduling_latency_s":0.789401,"redis_calls":13,"redis_duration_s":0.0018419999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001166,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010034,"completed_at":"2026-10-02T16:21:34.790Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.163642] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"448b469ef58096c6121ec48b","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-448b469ef58096c6121ec48b: start","job_status":"start","queue_duration_s":0.800346,"scheduling_latency_s":0.800359} gitlab # [ 266.174446] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"448b469ef58096c6121ec48b","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-448b469ef58096c6121ec48b: done: 0.009735 sec","job_status":"done","queue_duration_s":0.800346,"scheduling_latency_s":0.800359,"redis_calls":13,"redis_duration_s":0.001751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009735,"completed_at":"2026-10-02T16:21:34.800Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.231944] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95ca1fba7a2902b7b491373b","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95ca1fba7a2902b7b491373b: start","job_status":"start","queue_duration_s":0.812013,"scheduling_latency_s":0.812053} gitlab # [ 266.232807] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95ca1fba7a2902b7b491373b","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95ca1fba7a2902b7b491373b: done: 0.015064 sec","job_status":"done","queue_duration_s":0.812013,"scheduling_latency_s":0.812053,"redis_calls":13,"redis_duration_s":0.001928,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015064,"completed_at":"2026-10-02T16:21:34.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.279072] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39c158918ddc3471f5d398a0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39c158918ddc3471f5d398a0: start","job_status":"start","queue_duration_s":0.828021,"scheduling_latency_s":0.828036} gitlab # [ 266.279844] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39c158918ddc3471f5d398a0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39c158918ddc3471f5d398a0: done: 0.010498 sec","job_status":"done","queue_duration_s":0.828021,"scheduling_latency_s":0.828036,"redis_calls":13,"redis_duration_s":0.0017369999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010498,"completed_at":"2026-10-02T16:21:34.829Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.334248] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"110adec283b3b0558c8f61fa","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-110adec283b3b0558c8f61fa: start","job_status":"start","queue_duration_s":0.839451,"scheduling_latency_s":0.839466} gitlab # [ 266.334942] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"110adec283b3b0558c8f61fa","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-110adec283b3b0558c8f61fa: done: 0.010528 sec","job_status":"done","queue_duration_s":0.839451,"scheduling_latency_s":0.839466,"redis_calls":13,"redis_duration_s":0.0019979999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001455,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010528,"completed_at":"2026-10-02T16:21:34.840Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.387372] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0fe907efdab99d90cedc5cb","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0fe907efdab99d90cedc5cb: start","job_status":"start","queue_duration_s":0.851319,"scheduling_latency_s":0.851334} gitlab # [ 266.388214] gitaly[919]: time="2026-10-02T16:21:35.511Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQHNYVXTWEBXW07EESWBH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:22:05.333" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:35.433" grpc.time_ms=77.797 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.388917] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0fe907efdab99d90cedc5cb","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0fe907efdab99d90cedc5cb: done: 0.01033 sec","job_status":"done","queue_duration_s":0.851319,"scheduling_latency_s":0.851334,"redis_calls":13,"redis_duration_s":0.002133,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001497,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01033,"completed_at":"2026-10-02T16:21:34.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.433035] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a2129c0c17929db39543b28","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a2129c0c17929db39543b28: start","job_status":"start","queue_duration_s":0.862618,"scheduling_latency_s":0.862634} gitlab # [ 266.433708] gitaly[919]: time="2026-10-02T16:21:35.540Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQHNYVXTWEBXW07EESWBH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:05.439" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:35.539" grpc.time_ms=0.74 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.434521] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a2129c0c17929db39543b28","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a2129c0c17929db39543b28: done: 0.013307 sec","job_status":"done","queue_duration_s":0.862618,"scheduling_latency_s":0.862634,"redis_calls":13,"redis_duration_s":0.002095,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001441,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013307,"completed_at":"2026-10-02T16:21:34.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.486304] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8398e82dc28e8f9744da0ea0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8398e82dc28e8f9744da0ea0: start","job_status":"start","queue_duration_s":0.877539,"scheduling_latency_s":0.877561} gitlab # [ 266.486968] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8398e82dc28e8f9744da0ea0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8398e82dc28e8f9744da0ea0: done: 0.012129 sec","job_status":"done","queue_duration_s":0.877539,"scheduling_latency_s":0.877561,"redis_calls":13,"redis_duration_s":0.001941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001322,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012129,"completed_at":"2026-10-02T16:21:34.880Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.548740] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dbfcedeff449bc9be84ee7b","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dbfcedeff449bc9be84ee7b: start","job_status":"start","queue_duration_s":0.890692,"scheduling_latency_s":0.890709} gitlab # [ 266.549680] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dbfcedeff449bc9be84ee7b","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dbfcedeff449bc9be84ee7b: done: 0.010861 sec","job_status":"done","queue_duration_s":0.890692,"scheduling_latency_s":0.890709,"redis_calls":13,"redis_duration_s":0.001784,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010861,"completed_at":"2026-10-02T16:21:34.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.595274] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d2a6d53ca2ddc2995431b72","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d2a6d53ca2ddc2995431b72: start","job_status":"start","queue_duration_s":0.902524,"scheduling_latency_s":0.902537} gitlab # [ 266.596075] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d2a6d53ca2ddc2995431b72","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d2a6d53ca2ddc2995431b72: done: 0.010249 sec","job_status":"done","queue_duration_s":0.902524,"scheduling_latency_s":0.902537,"redis_calls":13,"redis_duration_s":0.001751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010249,"completed_at":"2026-10-02T16:21:34.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.637886] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d42efaae0c6445b956162789","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d42efaae0c6445b956162789: start","job_status":"start","queue_duration_s":0.913696,"scheduling_latency_s":0.91371} gitlab # [ 266.638759] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d42efaae0c6445b956162789","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d42efaae0c6445b956162789: done: 0.009971 sec","job_status":"done","queue_duration_s":0.913696,"scheduling_latency_s":0.91371,"redis_calls":13,"redis_duration_s":0.001724,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009971,"completed_at":"2026-10-02T16:21:34.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.688546] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af106a0cf18c947f75026527","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af106a0cf18c947f75026527: start","job_status":"start","queue_duration_s":0.924628,"scheduling_latency_s":0.924642} gitlab # [ 266.689483] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af106a0cf18c947f75026527","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af106a0cf18c947f75026527: done: 0.010287 sec","job_status":"done","queue_duration_s":0.924628,"scheduling_latency_s":0.924642,"redis_calls":13,"redis_duration_s":0.001768,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001205,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010287,"completed_at":"2026-10-02T16:21:34.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.734856] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64435b6085f31476ea035f14","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64435b6085f31476ea035f14: start","job_status":"start","queue_duration_s":0.93588,"scheduling_latency_s":0.935895} gitlab # [ 266.735789] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64435b6085f31476ea035f14","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64435b6085f31476ea035f14: done: 0.010275 sec","job_status":"done","queue_duration_s":0.93588,"scheduling_latency_s":0.935895,"redis_calls":13,"redis_duration_s":0.0017889999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010275,"completed_at":"2026-10-02T16:21:34.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.796069] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58da7714de3c8e3ae345df4a","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58da7714de3c8e3ae345df4a: start","job_status":"start","queue_duration_s":0.947358,"scheduling_latency_s":0.947374} gitlab # [ 266.796785] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58da7714de3c8e3ae345df4a","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58da7714de3c8e3ae345df4a: done: 0.010074 sec","job_status":"done","queue_duration_s":0.947358,"scheduling_latency_s":0.947374,"redis_calls":13,"redis_duration_s":0.001746,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010074,"completed_at":"2026-10-02T16:21:34.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.807513] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c43b62b084999958ba3f275","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c43b62b084999958ba3f275: start","job_status":"start","queue_duration_s":0.95834,"scheduling_latency_s":0.958354} gitlab # [ 266.818747] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c43b62b084999958ba3f275","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c43b62b084999958ba3f275: done: 0.010055 sec","job_status":"done","queue_duration_s":0.95834,"scheduling_latency_s":0.958354,"redis_calls":13,"redis_duration_s":0.00174,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010055,"completed_at":"2026-10-02T16:21:34.959Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.858686] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"726544bf6c58a34e16b4639d","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-726544bf6c58a34e16b4639d: start","job_status":"start","queue_duration_s":0.96959,"scheduling_latency_s":0.969604} gitlab # [ 266.865585] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"726544bf6c58a34e16b4639d","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-726544bf6c58a34e16b4639d: done: 0.009919 sec","job_status":"done","queue_duration_s":0.96959,"scheduling_latency_s":0.969604,"redis_calls":13,"redis_duration_s":0.001695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009919,"completed_at":"2026-10-02T16:21:34.970Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.916894] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c3513ef4d350846fe9ae58a","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c3513ef4d350846fe9ae58a: start","job_status":"start","queue_duration_s":0.980432,"scheduling_latency_s":0.980447} gitlab # [ 266.917643] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c3513ef4d350846fe9ae58a","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c3513ef4d350846fe9ae58a: done: 0.01066 sec","job_status":"done","queue_duration_s":0.980432,"scheduling_latency_s":0.980447,"redis_calls":13,"redis_duration_s":0.0023290000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001783,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01066,"completed_at":"2026-10-02T16:21:34.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.958682] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24d866ab999437d0ba09d8e6","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24d866ab999437d0ba09d8e6: start","job_status":"start","queue_duration_s":0.992026,"scheduling_latency_s":0.99223} gitlab # [ 266.960185] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24d866ab999437d0ba09d8e6","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24d866ab999437d0ba09d8e6: done: 0.010266 sec","job_status":"done","queue_duration_s":0.992026,"scheduling_latency_s":0.99223,"redis_calls":13,"redis_duration_s":0.001681,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010266,"completed_at":"2026-10-02T16:21:34.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.003359] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:34.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a04b9fc89f9ae496370a5792","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a04b9fc89f9ae496370a5792: start","job_status":"start","queue_duration_s":1.003218,"scheduling_latency_s":1.003232} gitlab # [ 267.005606] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a04b9fc89f9ae496370a5792","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a04b9fc89f9ae496370a5792: done: 0.010119 sec","job_status":"done","queue_duration_s":1.003218,"scheduling_latency_s":1.003232,"redis_calls":13,"redis_duration_s":0.001654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010119,"completed_at":"2026-10-02T16:21:35.004Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 267.070921] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbf135ac8c0b9d61c9e252b0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbf135ac8c0b9d61c9e252b0: start","job_status":"start","queue_duration_s":1.014287,"scheduling_latency_s":1.014301} gitlab # [ 267.071738] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbf135ac8c0b9d61c9e252b0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbf135ac8c0b9d61c9e252b0: done: 0.011623 sec","job_status":"done","queue_duration_s":1.014287,"scheduling_latency_s":1.014301,"redis_calls":13,"redis_duration_s":0.0028250000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001315,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011623,"completed_at":"2026-10-02T16:21:35.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.086756] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ad35d13a7f0590b846e4be1","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ad35d13a7f0590b846e4be1: start","job_status":"start","queue_duration_s":1.026861,"scheduling_latency_s":1.026876} gitlab # [ 267.097629] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ad35d13a7f0590b846e4be1","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ad35d13a7f0590b846e4be1: done: 0.010083 sec","job_status":"done","queue_duration_s":1.026861,"scheduling_latency_s":1.026876,"redis_calls":13,"redis_duration_s":0.0016849999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010083,"completed_at":"2026-10-02T16:21:35.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.09 seconds) gitlab # [ 267.141726] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c03b3292e21f23e16fb7ca14","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c03b3292e21f23e16fb7ca14: start","job_status":"start","queue_duration_s":1.038097,"scheduling_latency_s":1.038113} gitlab # [ 267.143123] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c03b3292e21f23e16fb7ca14","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c03b3292e21f23e16fb7ca14: done: 0.010406 sec","job_status":"done","queue_duration_s":1.038097,"scheduling_latency_s":1.038113,"redis_calls":13,"redis_duration_s":0.002186,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010406,"completed_at":"2026-10-02T16:21:35.039Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.185269] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5f035363889c55886107e95","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5f035363889c55886107e95: start","job_status":"start","queue_duration_s":1.049435,"scheduling_latency_s":1.049449} gitlab # [ 267.189677] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5f035363889c55886107e95","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5f035363889c55886107e95: done: 0.010163 sec","job_status":"done","queue_duration_s":1.049435,"scheduling_latency_s":1.049449,"redis_calls":13,"redis_duration_s":0.001715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010163,"completed_at":"2026-10-02T16:21:35.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.239687] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00b620db93a7385708bd7136","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00b620db93a7385708bd7136: start","job_status":"start","queue_duration_s":1.060704,"scheduling_latency_s":1.060718} gitlab # [ 267.240982] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 1313 "" "curl/8.21.0" 50 gitlab # [ 267.241793] gitaly[919]: time="2026-10-02T16:21:36.363Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQKY9DRGG47QPWRRNSM15 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:06.061" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:36.362" grpc.time_ms=0.696 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.279937] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00b620db93a7385708bd7136","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00b620db93a7385708bd7136: done: 0.00997 sec","job_status":"done","queue_duration_s":1.060704,"scheduling_latency_s":1.060718,"redis_calls":13,"redis_duration_s":0.0016320000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00997,"completed_at":"2026-10-02T16:21:35.061Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.283555] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b02c1a70c376b2e4e4a59e0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b02c1a70c376b2e4e4a59e0: start","job_status":"start","queue_duration_s":1.071582,"scheduling_latency_s":1.071596} gitlab # [ 267.294718] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b02c1a70c376b2e4e4a59e0","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b02c1a70c376b2e4e4a59e0: done: 0.009974 sec","job_status":"done","queue_duration_s":1.071582,"scheduling_latency_s":1.071596,"redis_calls":13,"redis_duration_s":0.0017100000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009974,"completed_at":"2026-10-02T16:21:35.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.344463] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"514bf536c3d21834aa97c96c","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-514bf536c3d21834aa97c96c: start","job_status":"start","queue_duration_s":1.082475,"scheduling_latency_s":1.082489} gitlab # [ 267.345256] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"514bf536c3d21834aa97c96c","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-514bf536c3d21834aa97c96c: done: 0.01022 sec","job_status":"done","queue_duration_s":1.082475,"scheduling_latency_s":1.082489,"redis_calls":13,"redis_duration_s":0.001676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01022,"completed_at":"2026-10-02T16:21:35.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.401562] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0af9c4c99231f93ff5314abc","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0af9c4c99231f93ff5314abc: start","job_status":"start","queue_duration_s":1.093606,"scheduling_latency_s":1.09362} gitlab # [ 267.402126] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0af9c4c99231f93ff5314abc","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0af9c4c99231f93ff5314abc: done: 0.010123 sec","job_status":"done","queue_duration_s":1.093606,"scheduling_latency_s":1.09362,"redis_calls":13,"redis_duration_s":0.001706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010123,"completed_at":"2026-10-02T16:21:35.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.445920] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80c09f2baac3e898c057569f","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80c09f2baac3e898c057569f: start","job_status":"start","queue_duration_s":1.10466,"scheduling_latency_s":1.104674} gitlab # [ 267.446726] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80c09f2baac3e898c057569f","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80c09f2baac3e898c057569f: done: 0.010728 sec","job_status":"done","queue_duration_s":1.10466,"scheduling_latency_s":1.104674,"redis_calls":13,"redis_duration_s":0.001667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010728,"completed_at":"2026-10-02T16:21:35.106Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.494340] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac9a1efd176974f1c418c927","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac9a1efd176974f1c418c927: start","job_status":"start","queue_duration_s":1.116291,"scheduling_latency_s":1.116306} gitlab # [ 267.495059] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac9a1efd176974f1c418c927","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac9a1efd176974f1c418c927: done: 0.010699 sec","job_status":"done","queue_duration_s":1.116291,"scheduling_latency_s":1.116306,"redis_calls":13,"redis_duration_s":0.001715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010699,"completed_at":"2026-10-02T16:21:35.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.513676] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2d1b32195c8ed8786fe7492","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2d1b32195c8ed8786fe7492: start","job_status":"start","queue_duration_s":1.127913,"scheduling_latency_s":1.127927} gitlab # [ 267.525236] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2d1b32195c8ed8786fe7492","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2d1b32195c8ed8786fe7492: done: 0.011071 sec","job_status":"done","queue_duration_s":1.127913,"scheduling_latency_s":1.127927,"redis_calls":13,"redis_duration_s":0.001679,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011071,"completed_at":"2026-10-02T16:21:35.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.566265] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31b921c5a101c5868e0fdb58","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31b921c5a101c5868e0fdb58: start","job_status":"start","queue_duration_s":1.139894,"scheduling_latency_s":1.139909} gitlab # [ 267.605888] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31b921c5a101c5868e0fdb58","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31b921c5a101c5868e0fdb58: done: 0.015853 sec","job_status":"done","queue_duration_s":1.139894,"scheduling_latency_s":1.139909,"redis_calls":13,"redis_duration_s":0.005337,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00233,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015853,"completed_at":"2026-10-02T16:21:35.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.607139] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12c40303c0235fcae57d66bd","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12c40303c0235fcae57d66bd: start","job_status":"start","queue_duration_s":1.156717,"scheduling_latency_s":1.156733} gitlab # [ 267.615776] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12c40303c0235fcae57d66bd","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12c40303c0235fcae57d66bd: done: 0.012659 sec","job_status":"done","queue_duration_s":1.156717,"scheduling_latency_s":1.156733,"redis_calls":13,"redis_duration_s":0.0019160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001358,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012659,"completed_at":"2026-10-02T16:21:35.160Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.670372] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5c031b0e23b898828989b04","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5c031b0e23b898828989b04: start","job_status":"start","queue_duration_s":1.170293,"scheduling_latency_s":1.170307} gitlab # [ 267.671134] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5c031b0e23b898828989b04","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5c031b0e23b898828989b04: done: 0.020012 sec","job_status":"done","queue_duration_s":1.170293,"scheduling_latency_s":1.170307,"redis_calls":13,"redis_duration_s":0.010272,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020012,"completed_at":"2026-10-02T16:21:35.180Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.725984] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c440abb5f743b0af27c4680","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c440abb5f743b0af27c4680: start","job_status":"start","queue_duration_s":1.194418,"scheduling_latency_s":1.194435} gitlab # [ 267.726744] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c440abb5f743b0af27c4680","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c440abb5f743b0af27c4680: done: 0.012659 sec","job_status":"done","queue_duration_s":1.194418,"scheduling_latency_s":1.194435,"redis_calls":13,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012659,"completed_at":"2026-10-02T16:21:35.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.772738] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7199d54980e676d108c5f3a3","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7199d54980e676d108c5f3a3: start","job_status":"start","queue_duration_s":1.272771,"scheduling_latency_s":1.272806} gitlab # [ 267.773604] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7199d54980e676d108c5f3a3","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7199d54980e676d108c5f3a3: done: 0.020159 sec","job_status":"done","queue_duration_s":1.272771,"scheduling_latency_s":1.272806,"redis_calls":13,"redis_duration_s":0.0065450000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000728,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005817,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020159,"completed_at":"2026-10-02T16:21:35.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.825314] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f97211905b59322925b1842b","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f97211905b59322925b1842b: start","job_status":"start","queue_duration_s":1.294312,"scheduling_latency_s":1.294329} gitlab # [ 267.826090] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f97211905b59322925b1842b","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f97211905b59322925b1842b: done: 0.011425 sec","job_status":"done","queue_duration_s":1.294312,"scheduling_latency_s":1.294329,"redis_calls":13,"redis_duration_s":0.0017859999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011425,"completed_at":"2026-10-02T16:21:35.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.865652] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0efcb2e4720df8c84676d05","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0efcb2e4720df8c84676d05: start","job_status":"start","queue_duration_s":1.307166,"scheduling_latency_s":1.30718} gitlab # [ 267.866167] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0efcb2e4720df8c84676d05","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0efcb2e4720df8c84676d05: done: 0.011125 sec","job_status":"done","queue_duration_s":1.307166,"scheduling_latency_s":1.30718,"redis_calls":13,"redis_duration_s":0.0017460000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011125,"completed_at":"2026-10-02T16:21:35.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.909855] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59f47fe414feb023367a9cd3","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59f47fe414feb023367a9cd3: start","job_status":"start","queue_duration_s":1.319536,"scheduling_latency_s":1.319551} gitlab # [ 267.910640] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59f47fe414feb023367a9cd3","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59f47fe414feb023367a9cd3: done: 0.010903 sec","job_status":"done","queue_duration_s":1.319536,"scheduling_latency_s":1.319551,"redis_calls":13,"redis_duration_s":0.001707,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010903,"completed_at":"2026-10-02T16:21:35.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.924942] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1e9eb5aca6ad58d979173f3","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1e9eb5aca6ad58d979173f3: start","job_status":"start","queue_duration_s":1.331372,"scheduling_latency_s":1.331387} gitlab # [ 267.963885] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1e9eb5aca6ad58d979173f3","created_at":"2026-10-02T16:21:33.852Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-84e3f14856fd48fa","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce0e5154f64bcde7f5e8cc4bf2f011f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:33.990Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1e9eb5aca6ad58d979173f3: done: 0.01141 sec","job_status":"done","queue_duration_s":1.331372,"scheduling_latency_s":1.331387,"redis_calls":13,"redis_duration_s":0.00212,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001535,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01141,"completed_at":"2026-10-02T16:21:35.333Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.970606] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"143d41efdcf01b704db8f3d3","created_at":"2026-10-02T16:21:33.999Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21e371c960cf6dc63ae372a3b7f7c97a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.010Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-143d41efdcf01b704db8f3d3: start","job_status":"start","queue_duration_s":1.324239,"scheduling_latency_s":1.324254} gitlab # [ 268.019686] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"143d41efdcf01b704db8f3d3","created_at":"2026-10-02T16:21:33.999Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21e371c960cf6dc63ae372a3b7f7c97a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.010Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-143d41efdcf01b704db8f3d3: done: 0.017076 sec","job_status":"done","queue_duration_s":1.324239,"scheduling_latency_s":1.324254,"redis_calls":11,"redis_duration_s":0.002017,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000603,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000292,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017076,"completed_at":"2026-10-02T16:21:35.351Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000803,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.020890] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.352Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ad14e1729f33e3abaeeda2db","created_at":"2026-10-02T16:21:34.013Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3c7925f2db11b0027d0c46a30f3adec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.026Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ad14e1729f33e3abaeeda2db: start","job_status":"start","queue_duration_s":1.325912,"scheduling_latency_s":1.325935} gitlab # [ 268.033722] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.362Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 268.071998] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ad14e1729f33e3abaeeda2db","created_at":"2026-10-02T16:21:34.013Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3c7925f2db11b0027d0c46a30f3adec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.026Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ad14e1729f33e3abaeeda2db: done: 0.015827 sec","job_status":"done","queue_duration_s":1.325912,"scheduling_latency_s":1.325935,"redis_calls":9,"redis_duration_s":0.003494,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015827,"completed_at":"2026-10-02T16:21:35.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.074033] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.369Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f5e033ca1b8667db8aac25a1","created_at":"2026-10-02T16:21:34.125Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-7b50c11bbfc64231","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.127Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f5e033ca1b8667db8aac25a1: start","job_status":"start","queue_duration_s":1.241924,"scheduling_latency_s":1.241941} gitlab # [ 268.088646] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.548Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f5e033ca1b8667db8aac25a1","created_at":"2026-10-02T16:21:34.125Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-7b50c11bbfc64231","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.127Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2573,"job_status":"rescheduled","message":"PipelineProcessWorker JID-f5e033ca1b8667db8aac25a1: rescheduled"} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses gitlab # [ 268.134771] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.554Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f5e033ca1b8667db8aac25a1","created_at":"2026-10-02T16:21:34.125Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-7b50c11bbfc64231","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.127Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f5e033ca1b8667db8aac25a1: done: 0.185138 sec","job_status":"done","queue_duration_s":1.241924,"scheduling_latency_s":1.241941,"gitaly_calls":2,"gitaly_duration_s":0.085773,"redis_calls":40,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006185,"redis_read_bytes":90,"redis_write_bytes":7682,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000938,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000854,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":6,"redis_queues_duration_s":0.000644,"redis_queues_read_bytes":9,"redis_queues_write_bytes":3278,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1187,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000172,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000337,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":29,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":14,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076976,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.15671014,"duration_s":0.185138,"completed_at":"2026-10-02T16:21:35.554Z","load_balancing_strategy":"primary","db_duration_s":0.022551,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.147888] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.556Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a5aa706dd1bc9e52899ff0dc","created_at":"2026-10-02T16:21:34.131Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9a102b6baf0d3f506cbcb62c1f5be58","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.151Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a5aa706dd1bc9e52899ff0dc: start","job_status":"start","queue_duration_s":1.40437,"scheduling_latency_s":1.404389} gitlab # [ 268.160691] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.569Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a5aa706dd1bc9e52899ff0dc","created_at":"2026-10-02T16:21:34.131Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9a102b6baf0d3f506cbcb62c1f5be58","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.151Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a5aa706dd1bc9e52899ff0dc: done: 0.013257 sec","job_status":"done","queue_duration_s":1.40437,"scheduling_latency_s":1.404389,"redis_calls":9,"redis_duration_s":0.0012959999999999998,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013257,"completed_at":"2026-10-02T16:21:35.569Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.209874] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c64037b15d871492dfdf210f","created_at":"2026-10-02T16:21:34.179Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82090fe40eb8ae2295a88f1ebb97b2aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.254Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c64037b15d871492dfdf210f: start","job_status":"start","queue_duration_s":1.316325,"scheduling_latency_s":1.316341} gitlab # [ 268.210618] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c64037b15d871492dfdf210f","created_at":"2026-10-02T16:21:34.179Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82090fe40eb8ae2295a88f1ebb97b2aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.254Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c64037b15d871492dfdf210f: done: 0.139859 sec","job_status":"done","queue_duration_s":1.316325,"scheduling_latency_s":1.316341,"redis_calls":13,"redis_duration_s":0.0021349999999999997,"redis_read_bytes":11,"redis_write_bytes":1563,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00057,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000767,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000337473,"exclusive_lock_hold_duration_s":0.124145666,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.139859,"completed_at":"2026-10-02T16:21:35.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.254205] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d37e3559f2b2a92c75bffb02","created_at":"2026-10-02T16:21:34.261Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"767798c8b4c59a22828873fe893b1c3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.272Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d37e3559f2b2a92c75bffb02: start","job_status":"start","queue_duration_s":1.439367,"scheduling_latency_s":1.439386} gitlab # [ 268.294197] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.719Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 268.294832] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d37e3559f2b2a92c75bffb02","created_at":"2026-10-02T16:21:34.261Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"767798c8b4c59a22828873fe893b1c3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.272Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d37e3559f2b2a92c75bffb02: done: 0.012814 sec","job_status":"done","queue_duration_s":1.439367,"scheduling_latency_s":1.439386,"redis_calls":9,"redis_duration_s":0.00129,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012814,"completed_at":"2026-10-02T16:21:35.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.303761] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.725Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1b8d3ff1a6ff05c6ade78b3f","created_at":"2026-10-02T16:21:34.287Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca75c3bfa474bb9ffc565cb9c9889ccf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.296Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1b8d3ff1a6ff05c6ade78b3f: start","job_status":"start","queue_duration_s":1.428509,"scheduling_latency_s":1.428537} gitlab # [ 268.317186] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.732Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 268.320415] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.736Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1b8d3ff1a6ff05c6ade78b3f","created_at":"2026-10-02T16:21:34.287Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca75c3bfa474bb9ffc565cb9c9889ccf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.296Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1b8d3ff1a6ff05c6ade78b3f: done: 0.011502 sec","job_status":"done","queue_duration_s":1.428509,"scheduling_latency_s":1.428537,"redis_calls":9,"redis_duration_s":0.001292,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011502,"completed_at":"2026-10-02T16:21:35.736Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000298,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.359626] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9079da6cf86432e61e5be5e8","created_at":"2026-10-02T16:21:34.321Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4aea35ffec19d0bc9215580e22a7fdaf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.351Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9079da6cf86432e61e5be5e8: start","job_status":"start","queue_duration_s":1.385987,"scheduling_latency_s":1.386003} gitlab # [ 268.398377] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9079da6cf86432e61e5be5e8","created_at":"2026-10-02T16:21:34.321Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4aea35ffec19d0bc9215580e22a7fdaf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.351Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9079da6cf86432e61e5be5e8: done: 0.024062 sec","job_status":"done","queue_duration_s":1.385987,"scheduling_latency_s":1.386003,"redis_calls":11,"redis_duration_s":0.002053,"redis_read_bytes":9,"redis_write_bytes":1509,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00038273,"exclusive_lock_hold_duration_s":0.010887417,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.024062,"completed_at":"2026-10-02T16:21:35.761Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.411906] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:35.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9a8abfeecc4b8eb105d51daf","created_at":"2026-10-02T16:21:34.359Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab80210fca08f2afbc247c0f2ed4f7cc","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.368Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9a8abfeecc4b8eb105d51daf: start","job_status":"start","queue_duration_s":1.396183,"scheduling_latency_s":1.396221} gitlab # [ 268.424761] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9a8abfeecc4b8eb105d51daf","created_at":"2026-10-02T16:21:34.359Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab80210fca08f2afbc247c0f2ed4f7cc","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.368Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9a8abfeecc4b8eb105d51daf: done: 0.260506 sec","job_status":"done","queue_duration_s":1.396183,"scheduling_latency_s":1.396221,"redis_calls":564,"redis_duration_s":0.08989,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.0887,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.193113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.260506,"completed_at":"2026-10-02T16:21:36.024Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000551,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.473368] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.025Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"1cc6cfa4ccf2170c6753d13c","created_at":"2026-10-02T16:21:34.381Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6166755924380138313e76738d90299","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.390Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-1cc6cfa4ccf2170c6753d13c: start","job_status":"start","queue_duration_s":1.635241,"scheduling_latency_s":1.635264} gitlab # [ 268.474448] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"1cc6cfa4ccf2170c6753d13c","created_at":"2026-10-02T16:21:34.381Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-069a568df10346c3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6166755924380138313e76738d90299","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.390Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-1cc6cfa4ccf2170c6753d13c: done: 0.016635 sec","job_status":"done","queue_duration_s":1.635241,"scheduling_latency_s":1.635264,"redis_calls":9,"redis_duration_s":0.0013700000000000001,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016635,"completed_at":"2026-10-02T16:21:36.042Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.513199] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.043Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5113086766c09031fe4edfb0","created_at":"2026-10-02T16:21:34.534Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-c6c59d7ef7fa49f3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.542Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5113086766c09031fe4edfb0: start","job_status":"start","queue_duration_s":1.500617,"scheduling_latency_s":1.500632} gitlab # [ 268.558053] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.056Z","message":"No existing merge request to be cleaned up."} gitlab # [ 268.558713] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.064Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5113086766c09031fe4edfb0","created_at":"2026-10-02T16:21:34.534Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-c6c59d7ef7fa49f3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.542Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5113086766c09031fe4edfb0: done: 0.021495 sec","job_status":"done","queue_duration_s":1.500617,"scheduling_latency_s":1.500632,"redis_calls":12,"redis_duration_s":0.002072,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001032,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021495,"completed_at":"2026-10-02T16:21:36.064Z","load_balancing_strategy":"primary","db_duration_s":0.002304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.561097] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.066Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"540fe953f6d7375217173d48","created_at":"2026-10-02T16:21:34.534Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-c6c59d7ef7fa49f3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.542Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-540fe953f6d7375217173d48: start","job_status":"start","queue_duration_s":1.523231,"scheduling_latency_s":1.523255} gitlab # [ 268.603788] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.073Z","message":"No existing merge request to be cleaned up."} gitlab # [ 268.604580] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.077Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"540fe953f6d7375217173d48","created_at":"2026-10-02T16:21:34.534Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-c6c59d7ef7fa49f3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.542Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-540fe953f6d7375217173d48: done: 0.011953 sec","job_status":"done","queue_duration_s":1.523231,"scheduling_latency_s":1.523255,"redis_calls":10,"redis_duration_s":0.001395,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011953,"completed_at":"2026-10-02T16:21:36.077Z","load_balancing_strategy":"primary","db_duration_s":0.001072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.608216] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.078Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6e996251e1fb6d5e3b5c15d1","created_at":"2026-10-02T16:21:34.534Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-c6c59d7ef7fa49f3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.542Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6e996251e1fb6d5e3b5c15d1: start","job_status":"start","queue_duration_s":1.536056,"scheduling_latency_s":1.536071} gitlab # [ 268.651961] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.085Z","message":"No existing merge request to be cleaned up."} gitlab # [ 268.652672] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.089Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6e996251e1fb6d5e3b5c15d1","created_at":"2026-10-02T16:21:34.534Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-c6c59d7ef7fa49f3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.542Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6e996251e1fb6d5e3b5c15d1: done: 0.011147 sec","job_status":"done","queue_duration_s":1.536056,"scheduling_latency_s":1.536071,"redis_calls":10,"redis_duration_s":0.001428,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000823,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011147,"completed_at":"2026-10-02T16:21:36.089Z","load_balancing_strategy":"primary","db_duration_s":0.001024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.654613] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.090Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4f1f3a40d002eac94fa87fd8","created_at":"2026-10-02T16:21:34.534Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-c6c59d7ef7fa49f3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.542Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4f1f3a40d002eac94fa87fd8: start","job_status":"start","queue_duration_s":1.548147,"scheduling_latency_s":1.548162} gitlab # [ 268.666029] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.097Z","message":"No existing merge request to be cleaned up."} gitlab # [ 268.667723] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.102Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4f1f3a40d002eac94fa87fd8","created_at":"2026-10-02T16:21:34.534Z","trace_propagation_headers":{"sentry-trace":"e998b304e0fe43f68e34a21c8592dc44-c6c59d7ef7fa49f3","baggage":"sentry-trace_id=e998b304e0fe43f68e34a21c8592dc44,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ee8e708f23a4c50aa8426420614b04f3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:34.542Z","job_size_bytes":2,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4f1f3a40d002eac94fa87fd8: done: 0.01115 sec","job_status":"done","queue_duration_s":1.548147,"scheduling_latency_s":1.548162,"redis_calls":10,"redis_duration_s":0.00148,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000615,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000865,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01115,"completed_at":"2026-10-02T16:21:36.102Z","load_balancing_strategy":"primary","db_duration_s":0.001072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.715770] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.103Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"459b2957e9e8a053c98199ea","created_at":"2026-10-02T16:21:35.419Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-9ad36b3b9bb84a22","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:35.421Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-459b2957e9e8a053c98199ea: start","job_status":"start","queue_duration_s":0.681355,"scheduling_latency_s":0.68137} gitlab # [ 268.716623] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.124Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"459b2957e9e8a053c98199ea","created_at":"2026-10-02T16:21:35.419Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-9ad36b3b9bb84a22","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:35.421Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-459b2957e9e8a053c98199ea: done: 0.021938 sec","job_status":"done","queue_duration_s":0.681355,"scheduling_latency_s":0.68137,"redis_calls":7,"redis_duration_s":0.00107,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021938,"completed_at":"2026-10-02T16:21:36.124Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.766203] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"cf419beb659a622264e73eb4","created_at":"2026-10-02T16:21:35.424Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-9ad36b3b9bb84a22","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:35.425Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-cf419beb659a622264e73eb4: start","job_status":"start","queue_duration_s":0.700205,"scheduling_latency_s":0.700221} gitlab # [ 268.804147] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.140Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"cf419beb659a622264e73eb4","created_at":"2026-10-02T16:21:35.424Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-9ad36b3b9bb84a22","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:35.425Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-cf419beb659a622264e73eb4: done: 0.014494 sec","job_status":"done","queue_duration_s":0.700205,"scheduling_latency_s":0.700221,"redis_calls":7,"redis_duration_s":0.0010509999999999999,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000307,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014494,"completed_at":"2026-10-02T16:21:36.140Z","load_balancing_strategy":"primary","db_duration_s":0.002045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.805225] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.141Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"7b7ccf2384b1a8edcc2318b0","created_at":"2026-10-02T16:21:35.549Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-9ad36b3b9bb84a22","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:35.550Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7b7ccf2384b1a8edcc2318b0: start","job_status":"start","queue_duration_s":0.590554,"scheduling_latency_s":0.590568} gitlab # [ 268.845970] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:36.157Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"7b7ccf2384b1a8edcc2318b0","created_at":"2026-10-02T16:21:35.549Z","trace_propagation_headers":{"sentry-trace":"c79f6e1dd3a448c8a74f967eadd39d33-9ad36b3b9bb84a22","baggage":"sentry-trace_id=c79f6e1dd3a448c8a74f967eadd39d33,sentry-release=Unknown"},"correlation_id":"01M3YPQHNYVXTWEBXW07EESWBH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:35.550Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7b7ccf2384b1a8edcc2318b0: done: 0.016116 sec","job_status":"done","queue_duration_s":0.590554,"scheduling_latency_s":0.590568,"redis_calls":8,"redis_duration_s":0.001368,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016116,"completed_at":"2026-10-02T16:21:36.157Z","load_balancing_strategy":"primary","db_duration_s":0.002289,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.961879] gitaly[919]: time="2026-10-02T16:21:38.161Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPQJ39XVRNEYB90T6ABCPH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:21:48.122" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:21:38.153" grpc.time_ms=8.557 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 268.994939] gitaly[919]: time="2026-10-02T16:21:38.195Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPQN0W7BQRP2868FEK2MAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:22:08.194" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:21:38.194" grpc.time_ms=0.749 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.049654] sidekiq-cluster[2573]: {"severity":"INFO","time":"2026-10-02T16:21:38.249Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"99c5382e4f3c716f0576449d","created_at":"2026-10-02T16:21:38.243Z","trace_propagation_headers":{"sentry-trace":"860fc57beec24822a867ffc69252e9d0-a5e1599de70b4b90","baggage":"sentry-trace_id=860fc57beec24822a867ffc69252e9d0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPQJ39XVRNEYB90T6ABCPH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:38.247Z","job_size_bytes":3,"pid":2573,"sidekiq_tid":"m82h","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-99c5382e4f3c716f0576449d: start","job_status":"start","queue_duration_s":0.002097,"scheduling_latency_s":0.002204} gitlab # [ 269.111645] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/763551d41e8de88b31a5ee89e2050c071ec59b45 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 3870 gitlab # [ 269.113228] nixbot[2696]: {"ts": "2026-10-02T16:21:38.313Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/763551d41e8de88b31a5ee89e2050c071ec59b45 \"HTTP/1.1 201 Created\""} gitlab # [ 269.116216] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 879 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/763551d41e8de88b31a5ee89e2050c071ec59b45/statuses, in 0.96 seconds) (finished: subtest: eval and build post commit statuses, in 10.06 seconds) (finished: run the VM test script, in 318.70 seconds) test script finished in 318.73s cleanup kill QemuMachine (pid 46) gitlab # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.50 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-02T16:20:38.028Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-02T16:21:27.831447Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] [] [{'id': 5, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:21:28.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:21:28.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:21:28.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:21:28.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:21:28.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-02T16:21:28.548Z', 'started_at': None, 'finished_at': '2026-10-02T16:21:34.118Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-10-02T16:21:34.300Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-02T16:21:28.548Z', 'started_at': None, 'finished_at': '2026-10-02T16:21:34.118Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-10-02T16:21:34.300Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-02T16:21:28.548Z', 'started_at': None, 'finished_at': '2026-10-02T16:21:34.118Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '763551d41e8de88b31a5ee89e2050c071ec59b45', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-10-02T16:21:34.300Z', 'started_at': None, 'finished_at': '2026-10-02T16:21:38.224Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]