tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 13) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.pqSi5KVAEB', fmt=raw size=1073741824 (finished: start all VMs, in 35.14 seconds) gitlab # mke2fs 1.47.3 (8-Jul-2025) gitlab: waiting for unit setup-gitlab.service gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for the VM to finish booting gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 80738cfb-64cd-4cd5-bb65-4e8e7f523a6b gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... ok gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/15f597533s2qv6mxxjladp3li5g5zvjg-nixos-system-gitlab-test/init regInfo=/nix/store/k6k50mqyg06zcznzgq5yid6bk6mjri5d-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000000] kvm-clock: using sched offset of 507489241 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.018 MHz processor gitlab # [ 0.000806] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000901] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002752] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002764] Using GB pages for direct mapping gitlab # [ 0.002824] RAMDISK: [mem 0xbe4e0000-0xbffcffff] gitlab # [ 0.002832] ACPI: Early table checksum verification disabled gitlab # [ 0.002835] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002839] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002843] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002850] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002852] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002853] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002855] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002856] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002858] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002859] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002860] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002860] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002861] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002861] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003355] No NUMA configuration found gitlab # [ 0.003357] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003360] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff] gitlab # [ 0.005824] Zone ranges: gitlab # [ 0.005825] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005827] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005828] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005830] Device empty gitlab # [ 0.005830] Movable zone start for each node gitlab # [ 0.005831] Early memory node ranges gitlab # [ 0.005831] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005832] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005833] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005834] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005854] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.006144] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.116704] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.117719] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.117737] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.117777] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.117780] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.117782] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.117783] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.117784] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.117785] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.117787] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.117789] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.117795] TSC deadline timer available gitlab # [ 0.117799] CPU topo: Max. logical packages: 1 gitlab # [ 0.117800] CPU topo: Max. logical dies: 1 gitlab # [ 0.117801] CPU topo: Max. dies per package: 1 gitlab # [ 0.117804] CPU topo: Max. threads per core: 1 gitlab # [ 0.117804] CPU topo: Num. cores per package: 4 gitlab # [ 0.117805] CPU topo: Num. threads per package: 4 gitlab # [ 0.117805] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.117833] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.117852] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.117855] kvm-guest: setup PV sched yield gitlab # [ 0.117875] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.117876] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.117877] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.117879] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.117881] Booting paravirtualized kernel on KVM gitlab # [ 0.117885] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.122371] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.124775] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.124827] kvm-guest: PV spinlocks enabled gitlab # [ 0.124829] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.124832] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/15f597533s2qv6mxxjladp3li5g5zvjg-nixos-system-gitlab-test/init regInfo=/nix/store/k6k50mqyg06zcznzgq5yid6bk6mjri5d-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.124932] Unknown kernel command line parameters "regInfo=/nix/store/k6k50mqyg06zcznzgq5yid6bk6mjri5d-closure-info/registration", will be passed to user space. gitlab # [ 0.124944] random: crng init done gitlab # [ 0.124945] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.133897] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.138297] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.138389] software IO TLB: area num 4. gitlab # [ 0.214618] Fallback order for Node 0: 0 gitlab # [ 0.214628] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.214630] Policy zone: Normal gitlab # [ 0.217590] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.227478] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.241366] allocated 12582912 bytes of page_ext gitlab # [ 0.251502] ftrace: allocating 48565 entries in 192 pages gitlab # [ 0.251504] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.252388] Dynamic Preempt: lazy gitlab # [ 0.252543] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.252544] rcu: RCU event tracing is enabled. gitlab # [ 0.252545] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.252546] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.252547] Rude variant of Tasks RCU enabled. gitlab # [ 0.252547] Tracing variant of Tasks RCU enabled. gitlab # [ 0.252548] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.252548] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.252568] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.252570] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.252571] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.257163] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.257453] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.257579] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.264799] Console: colour VGA+ 80x25 gitlab # [ 0.264802] printk: legacy console [tty0] enabled gitlab # [ 0.308346] printk: legacy console [ttyS0] enabled gitlab # [ 0.505161] ACPI: Core revision 20250807 gitlab # [ 0.506730] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.509575] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.511317] x2apic enabled gitlab # [ 0.512557] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.514257] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.516444] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.521344] kvm-guest: setup PV IPIs gitlab # [ 0.523854] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.525706] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229848b50c4, max_idle_ns: 440795322125 ns gitlab # [ 0.528799] Calibrating delay loop (skipped) preset value.. 4800.03 BogoMIPS (lpj=2400018) gitlab # [ 0.529915] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.531943] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.533798] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.535802] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.537798] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.538799] Transient Scheduler Attacks: Mitigation: Clear CPU buffers gitlab # [ 0.540797] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.542798] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.543798] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.544803] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.546798] active return thunk: srso_alias_return_thunk gitlab # [ 0.547826] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.548798] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.549797] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.550798] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.551797] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.552797] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.553798] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.555798] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers' gitlab # [ 0.556798] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)' gitlab # [ 0.558798] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.559797] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.560798] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.562798] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.564798] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.566798] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16 gitlab # [ 0.567798] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24 gitlab # [ 0.568798] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format. gitlab # [ 0.599022] Freeing SMP alternatives memory: 44K gitlab # [ 0.599801] pid_max: default: 32768 minimum: 301 gitlab # [ 0.600916] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.601913] landlock: Up and running. gitlab # [ 0.602798] Yama: becoming mindful. gitlab # [ 0.604011] LSM support for eBPF active gitlab # [ 0.606020] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.606945] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.610096] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.611391] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.611808] ... version: 2 gitlab # [ 0.612799] ... bit width: 48 gitlab # [ 0.613799] ... generic counters: 6 gitlab # [ 0.614799] ... generic bitmap: 000000000000003f gitlab # [ 0.615800] ... fixed-purpose counters: 0 gitlab # [ 0.616799] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.617800] ... value mask: 0000ffffffffffff gitlab # [ 0.618799] ... max period: 00007fffffffffff gitlab # [ 0.619799] ... global_ctrl mask: 000000000000003f gitlab # [ 0.620910] signal: max sigframe size: 3376 gitlab # [ 0.621888] rcu: Hierarchical SRCU implementation. gitlab # [ 0.622804] rcu: Max phase no-delay instances is 400. gitlab # [ 0.623997] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.629529] smp: Bringing up secondary CPUs ... gitlab # [ 0.631211] smpboot: x86: Booting SMP configuration: gitlab # [ 0.631810] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.635893] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.638803] smpboot: Total of 4 processors activated (19200.14 BogoMIPS) gitlab # [ 0.640207] Memory: 6013180K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved) gitlab # [ 0.642797] devtmpfs: initialized gitlab # [ 0.644028] x86/mm: Memory block size: 128MB gitlab # [ 0.647812] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.648853] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.649936] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.650926] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.652174] PM: RTC time: 06:48:49, date: 2026-07-09 gitlab # [ 0.655695] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.658128] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.658872] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.660905] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.661812] audit: initializing netlink subsys (disabled) gitlab # [ 0.663001] audit: type=2000 audit(1783579729.527:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.663081] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.664800] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.665800] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.666800] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.667799] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.668818] cpuidle: using governor menu gitlab # [ 0.672125] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.674117] PCI: Using configuration type 1 for base access gitlab # [ 0.674800] PCI: Using configuration type 1 for extended access gitlab # [ 0.676008] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.678848] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.679800] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.680800] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.681799] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.686605] ACPI: Added _OSI(Module Device) gitlab # [ 0.689809] ACPI: Added _OSI(Processor Device) gitlab # [ 0.693811] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.702923] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.712066] ACPI: Interpreter enabled gitlab # [ 0.714896] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.717821] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.722139] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.728809] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.733457] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.750730] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.753816] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.760062] acpiphp: Slot [3] registered gitlab # [ 0.762919] acpiphp: Slot [4] registered gitlab # [ 0.765920] acpiphp: Slot [5] registered gitlab # [ 0.767966] acpiphp: Slot [6] registered gitlab # [ 0.770910] acpiphp: Slot [7] registered gitlab # [ 0.772863] acpiphp: Slot [8] registered gitlab # [ 0.774861] acpiphp: Slot [9] registered gitlab # [ 0.775876] acpiphp: Slot [10] registered gitlab # [ 0.776877] acpiphp: Slot [11] registered gitlab # [ 0.777865] acpiphp: Slot [12] registered gitlab # [ 0.780104] acpiphp: Slot [13] registered gitlab # [ 0.781838] acpiphp: Slot [14] registered gitlab # [ 0.782840] acpiphp: Slot [15] registered gitlab # [ 0.783849] acpiphp: Slot [16] registered gitlab # [ 0.786908] acpiphp: Slot [17] registered gitlab # [ 0.789893] acpiphp: Slot [18] registered gitlab # [ 0.792890] acpiphp: Slot [19] registered gitlab # [ 0.795928] acpiphp: Slot [20] registered gitlab # [ 0.799893] acpiphp: Slot [21] registered gitlab # [ 0.802892] acpiphp: Slot [22] registered gitlab # [ 0.805891] acpiphp: Slot [23] registered gitlab # [ 0.808945] acpiphp: Slot [24] registered gitlab # [ 0.811893] acpiphp: Slot [25] registered gitlab # [ 0.814894] acpiphp: Slot [26] registered gitlab # [ 0.817872] acpiphp: Slot [27] registered gitlab # [ 0.819899] acpiphp: Slot [28] registered gitlab # [ 0.822941] acpiphp: Slot [29] registered gitlab # [ 0.824882] acpiphp: Slot [30] registered gitlab # [ 0.827905] acpiphp: Slot [31] registered gitlab # [ 0.829982] PCI host bridge to bus 0000:00 gitlab # [ 0.832821] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.835802] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.837803] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.839801] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.841810] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.843828] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.846324] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.851324] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.860133] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.870966] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.874986] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.879808] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.884808] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.889809] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.894591] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.904657] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.909654] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.916305] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.921847] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.926989] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 10742 usecs gitlab # [ 0.932924] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.955910] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.958827] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.960861] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.963064] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.976819] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 13671 usecs gitlab # [ 0.979995] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.991851] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.995846] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.999938] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 1.005854] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 1.016274] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 1.029843] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 1.032834] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 1.036908] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 1.045112] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 1.056838] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 1.059843] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 1.064938] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 1.076401] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 1.089856] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 1.094234] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 1.097938] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 1.107114] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.119853] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 1.121818] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 1.123852] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 1.128044] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.135137] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 1.136814] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 1.138852] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 1.142888] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 1.147844] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 1.150815] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 1.152860] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 1.154814] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 1.157850] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 1.163855] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 1.165852] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 1.169955] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 1.175852] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 1.176814] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 1.178853] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 1.185424] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 1.197844] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 1.201845] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 1.205938] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 1.229684] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 1.234235] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 1.238276] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 1.241178] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 1.245021] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 1.252231] iommu: Default domain type: Translated gitlab # [ 1.254804] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 1.259287] ACPI: bus type USB registered gitlab # [ 1.261967] usbcore: registered new interface driver usbfs gitlab # [ 1.265990] usbcore: registered new interface driver hub gitlab # [ 1.269876] usbcore: registered new device driver usb gitlab # [ 1.277069] NetLabel: Initializing gitlab # [ 1.279807] NetLabel: domain hash size = 128 gitlab # [ 1.282806] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 1.287039] NetLabel: unlabeled traffic allowed by default gitlab # [ 1.290841] PCI: Using ACPI for IRQ routing gitlab # [ 1.296014] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 1.296794] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 1.296794] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 1.307812] vgaarb: loaded gitlab # [ 1.309246] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 1.311799] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 1.317073] clocksource: Switched to clocksource kvm-clock gitlab # [ 1.323401] VFS: Disk quotas dquot_6.6.0 gitlab # [ 1.326543] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 1.332452] pnp: PnP ACPI init gitlab # [ 1.336696] pnp: PnP ACPI: found 6 devices gitlab # [ 1.352258] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 1.358761] clocksource: Switched to clocksource acpi_pm gitlab # [ 1.363291] NET: Registered PF_INET protocol family gitlab # [ 1.368996] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 1.395504] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 1.402251] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 1.408518] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 1.420574] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 1.426528] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 1.432658] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 1.439208] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.445115] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.450559] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 1.454011] NET: Registered PF_XDP protocol family gitlab # [ 1.457104] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.460823] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.464527] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.467366] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.469576] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.475595] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.480024] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.488786] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.496876] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x7c0 took 11977 usecs gitlab # [ 1.502917] PCI: CLS 0 bytes, default 64 gitlab # [ 1.506677] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.507137] Trying to unpack rootfs image as initramfs... gitlab # [ 1.507611] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB) gitlab # [ 1.521676] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229848b50c4, max_idle_ns: 440795322125 ns gitlab # [ 1.569888] Initialise system trusted keyrings gitlab # [ 1.573283] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.597425] Key type asymmetric registered gitlab # [ 1.598742] Asymmetric key parser 'x509' registered gitlab # [ 1.600305] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.603031] io scheduler mq-deadline registered gitlab # [ 1.604462] io scheduler kyber registered gitlab # [ 1.609949] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.612196] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.616212] Linux agpgart interface v0.103 gitlab # [ 1.617691] ACPI: bus type drm_connector registered gitlab # [ 1.621924] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.623856] usbserial: USB Serial support registered for generic gitlab # [ 1.625637] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.625637] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.629557] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.631919] drop_monitor: Initializing network drop monitor service gitlab # [ 1.633949] NET: Registered PF_INET6 protocol family gitlab # [ 1.636585] Segment Routing with IPv6 gitlab # [ 1.637856] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.641148] IPI shorthand broadcast: enabled gitlab # [ 1.646389] sched_clock: Marking stable (1387023811, 259332325)->(1730072274, -83716138) gitlab # [ 1.649390] registered taskstats version 1 gitlab # [ 1.651016] Loading compiled-in X.509 certificates gitlab # [ 1.661850] Demotion targets for Node 0: null gitlab # [ 1.663766] Key type .fscrypt registered gitlab # [ 1.665062] Key type fscrypt-provisioning registered gitlab # [ 1.666798] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.668501] ima: Allocated hash algorithm: sha1 gitlab # [ 1.669976] ima: No architecture policies found gitlab # [ 1.671794] PM: Magic number: 2:105:822 gitlab # [ 1.674129] RAS: Correctable Errors collector initialized. gitlab # [ 1.679579] clk: Disabling unused clocks gitlab # [ 1.680858] PM: genpd: Disabling unused power domains gitlab # [ 1.792816] Freeing initrd memory: 27584K gitlab # [ 1.797477] Freeing unused decrypted memory: 2028K gitlab # [ 1.801266] Freeing unused kernel image (initmem) memory: 3644K gitlab # [ 1.803193] Write protecting the kernel read-only data: 32768k gitlab # [ 1.806423] Freeing unused kernel image (text/rodata gap) memory: 1332K gitlab # [ 1.809045] Freeing unused kernel image (rodata/data gap) memory: 816K gitlab # [ 1.861463] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.863510] Run /init as init process gitlab # [ 1.877713] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.905083] fuse: init (API version 7.45) gitlab # [ 1.928453] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.938950] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.944001] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.997433] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 2.338078] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 2.359670] systemd[1]: Detected virtualization kvm. gitlab # [ 2.363648] systemd[1]: Detected architecture x86-64. gitlab # [ 2.367624] systemd[1]: Running in initrd. gitlab # [ 2.371830] systemd[1]: Initializing machine ID from random generator. gitlab # [ 2.377059] systemd[1]: Hostname set to . gitlab # [ 2.455208] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.634505] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.639557] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.645643] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.650707] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.655677] systemd[1]: Reached target Path Units. gitlab # [ 2.659702] systemd[1]: Reached target Slice Units. gitlab # [ 2.663708] systemd[1]: Reached target Swaps. gitlab # [ 2.667470] systemd[1]: Reached target Timer Units. gitlab # [ 2.670883] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.675165] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.679281] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.682883] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.685026] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.686756] systemd[1]: Reached target Socket Units. gitlab # [ 2.693645] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.700193] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.712904] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.718147] systemd[1]: Starting Journal Service... gitlab # [ 2.727954] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.733652] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 2.737627] systemd-journald[152]: Collecting audit messages is disabled. gitlab # [ 2.756948] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.780928] netfs: FS-Cache loaded gitlab # [ 2.785103] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.791045] 9pnet: Installing 9P2000 support gitlab # [ 2.791182] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.799481] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.799956] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.806994] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.808747] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.813776] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.820272] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.822858] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.828980] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.571588] systemd-modules-load[155]: Using 4 probe threads gitlab # [ 2.573738] systemd-modules-l[ 2.832836] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # oad[155]: Inserted module 'virtio_balloon' gitlab # [ 2.577041] systemd-modules-load[155]: Inserted module 'virtio_gpu' gitlab # [ 2.837633] systemd[1]: Started Journal Service. gitlab # [ 2.578826] systemd-modules-load[155]: Inserted module 'dm_mod' gitlab # [ 2.591398] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.595647] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.600470] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.625064] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.626641] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.628229] systemd[1]: Reached target Local File Systems. gitlab # [ 2.629754] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.632380] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.655495] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.661188] systemd-udevd[192]: Using default interface naming scheme 'v260'. gitlab # [ 2.677570] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.693195] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.694567] systemd[1]: Reached target System Initialization. gitlab # [ 2.695952] systemd[1]: Reached target Basic System. gitlab # [ 3.116066] SCSI subsystem initialized gitlab # [ 3.131250] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 3.134396] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 3.135754] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 3.142718] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 3.146024] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 3.150462] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 3.154959] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 3.156978] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 3.159013] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 3.160767] usb usb1: Product: UHCI Host Controller gitlab # [ 3.161245] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 3.162036] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd gitlab # [ 3.164938] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 3.169893] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 3.175758] hub 1-0:1.0: USB hub found gitlab # [ 3.176923] hub 1-0:1.0: 2 ports detected gitlab # [ 3.181974] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 3.190610] virtio_blk virtio5: [vdb] 11450728 512-byte logical blocks (5.86 GB/5.46 GiB) gitlab # [ 3.210607] scsi host0: ata_piix gitlab # [ 3.214607] scsi host1: ata_piix gitlab # [ 3.215663] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 3.217475] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.966599] (udev-worker)[215]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.971759] (udev-worker)[215]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.982290] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.986470] (udev-worker)[220]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.989746] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.992482] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.993920] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.995458] systemd-vconsole-setup[231]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.998227] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 3.014949] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.031581] systemd-fsck[240]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 3.038162] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 3.373738] ata2: found unknown device (class 0) gitlab # [ 3.378576] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 3.385883] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 3.406462] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 3.428148] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 3.473675] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 3.504608] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 3.577633] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 3.582465] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 3.587585] usb 1-1: Product: QEMU USB Tablet gitlab # [ 3.590667] usb 1-1: Manufacturer: QEMU gitlab # [ 3.593268] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 3.635573] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 3.649561] usbcore: registered new interface driver usbhid gitlab # [ 3.652167] usbhid: USB HID core driver gitlab # [ 3.664265] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 3.668645] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 3.528751] systemd[1]: Mounting /sysroot... gitlab # [ 3.966505] EXT4-fs (vda): mounted filesystem 80738cfb-64cd-4cd5-bb65-4e8e7f523a6b r/w with ordered data mode. Quota mode: none. gitlab # [ 3.713299] systemd[1]: Mounted /sysroot. gitlab # [ 3.715387] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.721412] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.725537] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.735337] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.740160] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 4.004015] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.745436] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.747789] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.755123] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.756473] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.757767] systemd[1]: Mounted /sysroot/run. gitlab # [ 4.019827] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.761069] systemd-sysroot-fstab-check[269]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.765812] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.767869] systemd[1]: Reload requested from client PID 269 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.770515] systemd[1]: Reloading... gitlab # [ 4.030394] systemd[1]: Reloading finished in 263 ms. gitlab # [ 4.087881] systemd-sysroot-fstab-check[269]: Requesting initrd-fs.target/start/replace... gitlab # [ 4.233358] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 4.236810] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 4.240563] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.246008] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.260190] systemd-sysroot-fstab-check[269]: Requesting swap.target/start/replace... gitlab # [ 4.271532] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 4.280008] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 4.284426] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 4.298353] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.324365] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.332962] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.529754] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.584918] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.588755] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.594642] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.600210] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.635291] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.641799] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.659250] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.664550] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.672216] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.714686] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.718364] systemd[1]: Stopped target Basic System. gitlab # [ 4.721563] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.724987] systemd[1]: Stopped target Path Units. gitlab # [ 4.728187] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.732632] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.737199] systemd[1]: Stopped target Slice Units. gitlab # [ 4.740379] systemd[1]: Stopped target Socket Units. gitlab # [ 4.743530] systemd[1]: Stopped target System Initialization. gitlab # [ 4.746973] systemd[1]: Stopped target Swaps. gitlab # [ 4.749828] systemd[1]: Stopped target Timer Units. gitlab # [ 4.752686] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.755643] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.758448] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.761875] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.764254] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.769101] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.771513] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.773194] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.774488] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.776256] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.778822] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.780739] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.786525] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.788276] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.791618] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.793005] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.794635] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.796328] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.798843] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.800626] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.802794] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.811437] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.816484] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.822246] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.827277] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.829065] systemd[1]: systemd-udevd.service: Consumed 1.108s CPU time over 2.190s wall clock time, 26.2M memory peak. gitlab # [ 4.838414] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.841725] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.849616] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.851349] systemd[1]: Closed udev Control Socket. gitlab # [ 4.854367] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.855752] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.857566] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.859089] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.861055] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.862812] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.864511] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.901459] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.907462] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.916509] systemd[1]: Reached target Switch Root. gitlab # [ 4.922498] systemd[1]: Starting NixOS Activation... gitlab # [ 5.067613] initrd-nixos-activation-start[570]: booting system configuration /nix/store/15f597533s2qv6mxxjladp3li5g5zvjg-nixos-system-gitlab-test gitlab # [ 5.115917] initrd-nixos-activation-start[570]: running activation script... gitlab # [ 5.332978] initrd-nixos-activation-start[593]: setting up /etc... gitlab # [ 5.475273] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 5.482507] systemd[1]: Finished NixOS Activation. gitlab # [ 5.498684] systemd[1]: Starting Switch Root... gitlab # [ 5.540496] systemd[1]: Switching root. gitlab # [ 6.235825] systemd-journald[152]: Received SIGTERM from PID 1 (systemd). gitlab # [ 6.358724] NET: Registered PF_VSOCK protocol family gitlab # [ 6.728172] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 6.744175] systemd[1]: Detected virtualization kvm. gitlab # [ 6.747288] systemd[1]: Detected architecture x86-64. gitlab # [ 6.750649] systemd[1]: Detected first boot. gitlab # [ 6.754798] systemd[1]: Initializing machine ID from random generator. gitlab # [ 6.894475] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 6.958877] systemd[1]: Applying preset policy. gitlab # [ 7.136086] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 7.262984] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 7.265346] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 7.268888] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 7.272289] systemd[1]: Created slice Slice /system/getty. gitlab # [ 7.274503] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 7.276813] systemd[1]: Created slice User and Session Slice. gitlab # [ 7.278410] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 7.280501] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 7.282374] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 7.283698] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 7.285037] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 7.286556] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 7.287933] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 7.289387] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 7.290941] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 7.292617] systemd[1]: Reached target Path Units. gitlab # [ 7.293909] systemd[1]: Reached target Remote File Systems. gitlab # [ 7.295370] systemd[1]: Reached target Slice Units. gitlab # [ 7.296663] systemd[1]: Reached target Swaps. gitlab # [ 7.298924] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 7.301008] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 7.304497] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 7.306686] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 7.308372] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 7.310238] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.312699] systemd[1]: Listening on udev Control Socket. gitlab # [ 7.317790] systemd[1]: Mounting Huge Pages File System... gitlab # [ 7.320719] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 7.323715] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 7.327429] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 7.336240] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 7.338941] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 7.344121] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 7.348297] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 7.354599] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.354794] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 7.361428] systemd[1]: Mounting FUSE Control File System... gitlab # [ 7.365540] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.386441] systemd[1]: Starting Journal Service... gitlab # [ 7.390522] systemd[1]: Starting Load Kernel Modules... gitlab # [ 7.395373] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 7.401506] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 7.403215] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.411467] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 7.417279] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 7.419086] systemd[1]: Mounted Huge Pages File System. gitlab # [ 7.419704] systemd-journald[805]: Collecting audit messages is enabled. gitlab # [ 7.421174] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 7.424600] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 7.424825] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 7.428499] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 7.431240] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 7.434446] EXT4-fs (vda): re-mounted 80738cfb-64cd-4cd5-bb65-4e8e7f523a6b. gitlab # [ 7.439896] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 7.183440] systemd[1]: Queue[ 7.442639] loop: module loaded gitlab # d start job for default target Multi-User System. gitlab # [ 7.186485] systemd[1]: Unnecessary job was r[ 7.445885] systemd[1]: Started Journal Service. gitlab # emoved for /dev/vdb. gitlab # [ 7.189436] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 7.191863] systemd-modules-load[806]: Using 3 probe threads gitlab # [ 7.193601] systemd-modules-load[806]: Inserted module 'tls' gitlab # [ 7.195415] systemd-modules-load[806]: Inserted module 'loop' gitlab # [ 7.197141] systemd[1]: Mounted FUSE Control File System. gitlab # [ 7.199349] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 7.201296] systemd[1]: Finished Load Kernel Modules. gitlab # [ 7.204568] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 7.206425] systemd[1]: Starting Firewall... gitlab # [ 7.209428] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 7.212205] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.219258] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 7.223081] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 7.224988] systemd-oomd[807]: No swap; memory pressure usage will be degraded gitlab # [ 7.227969] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 7.232101] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.494384] systemd-journald[805]: Received client request to flush runtime journal. gitlab # [ 7.308567] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 7.311282] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 7.313080] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 7.315870] systemd[1]: Reached target First Boot Complete. gitlab # [ 7.317532] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 7.319082] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 7.321209] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 7.322879] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 7.324406] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 7.326097] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 7.328057] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 7.339308] systemd-udevd[854]: Using default interface naming scheme 'v260'. gitlab # [ 7.378279] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 7.385371] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 7.500123] systemd[1]: Finished Firewall. gitlab # [ 7.502693] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 7.532797] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 7.544919] (udev-worker)[941]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 7.548832] (udev-worker)[941]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.551265] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 7.554057] (udev-worker)[944]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.565777] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 7.568913] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.572161] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.577323] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.580229] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.583582] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.878879] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 7.880829] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 7.884583] mousedev: PS/2 mouse device common for all mice gitlab # [ 7.884766] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 7.903548] ACPI: button: Power Button [PWRF] gitlab # [ 7.940675] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 7.944100] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 7.945118] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 7.951412] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 7.951595] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 7.954103] FDC 0 is a S82078B gitlab # [ 7.955530] rtc_cmos 00:05: registered as rtc0 gitlab # [ 7.955702] rtc_cmos 00:05: setting system clock to 2026-07-09T06:48:56 UTC (1783579736) gitlab # [ 7.955881] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 7.959124] systemd-journald[805]: Time jumped backwards, rotating. gitlab # [ 7.962126] Console: switching to colour dummy device 80x25 gitlab # [ 7.973534] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 7.976376] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 7.982430] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 7.984208] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 7.984241] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 7.987257] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 7.996473] fbcon: bochs-drmdrmfb (fb0) is primary device gitlab # [ 8.002934] Console: switching to colour frame buffer device 160x50 gitlab # [ 8.006113] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 8.010799] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 7.811121] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.827868] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.832426] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.859339] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.873092] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.875126] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.881908] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.158662] ppdev: user-space parallel port driver gitlab # [ 8.185742] kvm_amd: TSC scaling supported gitlab # [ 8.186883] kvm_amd: Nested Virtualization enabled gitlab # [ 8.188436] kvm_amd: Nested Paging enabled gitlab # [ 8.189587] kvm_amd: LBR virtualization supported gitlab # [ 8.190767] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 8.192083] kvm_amd: Virtual GIF supported gitlab # [ 8.193081] kvm_amd: Virtual NMI enabled gitlab # [ 8.242843] EDAC MC: Ver: 3.0.0 gitlab # [ 7.999724] systemd-vconsole-setup[978]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 8.005043] systemd[1]: Finished Virtual Console Setup. gitlab # [ 8.007745] systemd[1]: Mounting /run/wrappers... gitlab # [ 8.029799] systemd[1]: Mounted /run/wrappers. gitlab # [ 8.031148] systemd[1]: Reached target Local File Systems. gitlab # [ 8.032607] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 8.034378] systemd[1]: Starting register-nix-paths.service... gitlab # [ 8.035988] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 8.037628] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 8.039978] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 8.044869] systemd[1]: Starting Create System Files and Directories... gitlab # [ 8.065889] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 8.069385] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 8.077703] systemd[1]: Finished Create System Files and Directories. gitlab # [ 8.080101] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 8.082429] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 8.104951] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 8.117488] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 8.121893] systemd[1]: Starting Update is Completed... gitlab # [ 8.139544] systemd[1]: Finished Update is Completed. gitlab # [ 8.182403] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 8.184285] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 8.185649] systemd[1]: suid-sgid-wrappers.service: Consumed 120ms CPU time over 147ms wall clock time, 5M memory peak, 1.1M read from disk. gitlab # [ 8.446802] systemd[1]: Finished register-nix-paths.service. gitlab # [ 8.449457] systemd[1]: Reached target System Initialization. gitlab # [ 8.451700] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 8.454336] systemd[1]: Started logrotate.timer. gitlab # [ 8.456474] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 8.458788] systemd[1]: Reached target Timer Units. gitlab # [ 8.460791] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 8.463363] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 8.468768] systemd[1]: Starting nixbot CI service socket... gitlab # [ 8.471141] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 8.473503] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 8.475885] systemd[1]: Reached target Socket Units. gitlab # [ 8.478442] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 8.546269] dbus-broker-launch[1070]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 8.551691] dbus-broker-launch[1070]: NSS returned no entry for 'systemd-timesync' gitlab # [ 8.556605] dbus-broker-launch[1070]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 8.565574] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 8.578703] dbus-broker-launch[1070]: Ready gitlab # [ 8.584890] systemd[1]: Reached target Basic System. gitlab # [ 8.588177] systemd[1]: Started backdoor.service. gitlab # [ 8.591291] systemd[1]: Starting gitlab-config.service... gitlab # [ 8.594386] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 8.599693] systemd[1]: Starting Logrotate configuration check... gitlab # [ 8.606368] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.611963] systemd[1]: Starting Post-Boot Actions... gitlab # [ 8.618510] systemd[1]: Started Reset console on configuration changes. gitlab # [ 8.622064] logrotate[1075]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 8.630360] logrotate[1075]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf gitlab # connecting to host... gitlab # [ 8.634036] logrotate[1075]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 8.636804] logrotate[1075]: Reading state from file: /var/lib/logrotate.status gitlab # [ 8.638790] logrotate[1075]: state file /var/lib/logrotate.status does not exist gitlab # [ 8.640692] logrotate[1075]: Allocating hash table for state file, size 64 entries gitlab # [ 8.642738] logrotate[1075]: Handling 4 logs gitlab # [ 8.643894] logrotate[1075]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab: connected to guest root shell gitlab: (connecting took 9.50 seconds) gitlab: (finished: waiting for the VM to finish booting, in 9.50 seconds) gitlab # [ 8.647504] logrotate[1075]: considering log /var/log/btmp gitlab # [ 8.648933] logrotate[1075]: Creating new state gitlab # [ 8.650973] logrotate[1075]: Now: 2026-07-09 06:48 gitlab # [ 8.652224] logrotate[1075]: Last rotated at 2026-07-09 06:00 gitlab # [ 8.655603] logrotate[1075]: log does not need rotating (log has already been rotated) gitlab # [ 8.656853] logrotate[1075]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 8.658649] logrotate[1075]: considering log /var/log/wtmp gitlab # [ 8.659770] logrotate[1075]: Creating new state gitlab # [ 8.661228] logrotate[1075]: Now: 2026-07-09 06:48 gitlab # [ 8.662819] logrotate[1075]: Last rotated at 2026-07-09 06:00 gitlab # [ 8.663982] logrotate[1075]: log does not need rotating (log has already been rotated) gitlab # [ 8.666933] logrotate[1075]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 8.668812] logrotate[1075]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1075) gitlab # [ 8.669609] logrotate[1075]: considering log /var/gitlab/state/log/*.log gitlab # [ 8.670474] logrotate[1075]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 8.671635] logrotate[1075]: Creating new state gitlab # [ 8.673136] logrotate[1075]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1075) gitlab # [ 8.674042] logrotate[1075]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 8.674586] logrotate[1075]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1075) gitlab # [ 8.675707] logrotate[1075]: considering log /var/log/nginx/*.log gitlab # [ 8.677205] logrotate[1075]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 8.677992] logrotate[1075]: Creating new state gitlab # [ 8.679192] logrotate[1075]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1075) gitlab # [ 8.680187] systemd[1]: Starting resolvconf update... gitlab # [ 8.681426] nsncd[1076]: Jul 09 06:48:57.466 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.683131] systemd[1]: Finished Logrotate configuration check. gitlab # [ 8.684952] systemd[1]: Finished Post-Boot Actions. gitlab # [ 8.687593] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 8.688816] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.690330] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.692325] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.693112] systemd[1]: Starting User Login Management... gitlab # [ 8.707935] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 8.709208] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 8.710473] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 8.711400] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 8.713639] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 8.715794] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 8.718142] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 8.719811] systemd[1]: nscd.service: Consumed 21ms CPU time over 110ms wall clock time, 7.8M memory peak, 5.6M read from disk. gitlab # [ 8.732306] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.752737] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.754175] nsncd[1156]: Jul 09 06:48:57.556 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.756934] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.758187] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.761958] systemd-logind[1118]: New seat seat0. gitlab # [ 8.767171] systemd-logind[1118]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 8.768386] systemd-logind[1118]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 8.769404] systemd-logind[1118]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 8.776329] systemd[1]: Started User Login Management. gitlab # [ 8.777565] systemd[1]: Starting linger-users.service... gitlab # [ 8.778908] systemd[1]: Finished resolvconf update. gitlab # [ 8.779685] systemd[1]: Reached target Preparation for Network. gitlab # [ 8.785695] systemd[1]: Starting DHCP Client... gitlab # [ 8.789435] systemd[1]: Starting Address configuration of eth1... gitlab # [ 8.791315] systemd[1]: Starting Extra networking commands.... gitlab # [ 8.794274] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 8.800946] systemd[1]: Finished linger-users.service. gitlab # [ 8.824607] network-addresses-eth1-start[1187]: adding address 192.168.1.1/24... done gitlab # [ 8.830161] network-addresses-eth1-start[1187]: adding address 2001:db8:1::1/64... done gitlab # [ 8.836640] systemd[1]: Finished Address configuration of eth1. gitlab # [ 8.869761] systemd[1]: Finished Extra networking commands.. gitlab # [ 8.871601] systemd[1]: Reached target Network. gitlab # [ 8.876809] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 8.879639] dhcpcd[1202]: dhcpcd-10.3.2 starting gitlab # [ 8.882344] systemd[1]: Starting Nginx Web Server... gitlab # [ 8.885766] dhcpcd[1248]: dev: loaded udev gitlab # [ 8.887053] systemd[1]: Starting PostgreSQL Server... gitlab # [ 8.891102] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 9.152811] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 9.153773] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.898326] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.913292] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.918486] systemd[1]: Finished Permit User Sessions. gitlab # [ 8.924874] systemd[1]: Started Getty on tty1. gitlab # [ 8.927866] systemd[1]: Reached target Login Prompts. gitlab # [ 9.254703] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 9.265225] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 9.267383] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 9.269623] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 9.271472] cfg80211: failed to load regulatory.db gitlab # [ 9.027233] nginx-pre-start[1268]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok gitlab # [ 9.029690] nginx-pre-start[1268]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful gitlab # [ 9.037463] systemd[1]: Started Nginx Web Server. gitlab # [ 9.050241] redis-server[1270]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 9.051956] redis-server[1270]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 9.057335] redis-server[1270]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1270, just started gitlab # [ 9.058775] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 9.059940] redis-server[1270]: Configuration loaded gitlab # [ 9.061082] redis[1270]: monotonic clock: POSIX clock_gettime gitlab # [ 9.061919] redis[1270]: Running mode=standalone, port=0. gitlab # [ 9.063172] redis[1270]: Server initialized gitlab # [ 9.064193] redis[1270]: Ready to accept connections unix gitlab # [ 9.065070] redis[1270]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 9.330716] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 9.073759] dhcpcd[1248]: eth0: waiting for carrier gitlab # [ 9.074891] dhcpcd[1248]: eth0: carrier acquired gitlab # [ 9.080181] dhcpcd[1248]: DUID 00:01:00:01:31:e2:00:d9:52:54:00:12:34:56 gitlab # [ 9.081490] dhcpcd[1248]: eth0: IAID 00:12:34:56 gitlab # [ 9.083400] dhcpcd[1248]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 9.111953] postgresql-pre-start[1277]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 9.113370] postgresql-pre-start[1277]: This user must also own the server process. gitlab # [ 9.116195] postgresql-pre-start[1277]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 9.116957] postgresql-pre-start[1277]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 9.118130] postgresql-pre-start[1277]: The default text search configuration will be set to "english". gitlab # [ 9.119047] postgresql-pre-start[1277]: Data page checksums are disabled. gitlab # [ 9.120131] postgresql-pre-start[1277]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok gitlab # [ 9.121395] postgresql-pre-start[1277]: creating subdirectories ... ok gitlab # [ 9.122819] postgresql-pre-start[1277]: selecting dynamic shared memory implementation ... posix gitlab # [ 9.146608] postgresql-pre-start[1277]: selecting default "max_connections" ... 100 gitlab # [ 9.192846] postgresql-pre-start[1277]: selecting default "shared_buffers" ... 128MB gitlab # [ 9.383305] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 9.584381] postgresql-pre-start[1277]: selecting default time zone ... UTC gitlab # [ 9.586711] postgresql-pre-start[1277]: creating configuration files ... ok gitlab # [ 9.668715] gitlab-workhorse[1306]: time="2026-07-09T06:48:58Z" level=info msg=Starting build_time=19700101.000000 version=18.11.4 gitlab # [ 9.674412] gitlab-workhorse[1306]: time="2026-07-09T06:48:58Z" level=info msg="Using redis/go-redis" gitlab # [ 9.679110] gitlab-workhorse[1306]: time="2026-07-09T06:48:58Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 9.772993] postgresql-pre-start[1277]: running bootstrap script ... ok gitlab # [ 9.799337] dhcpcd[1248]: eth0: soliciting a DHCP lease gitlab # [ 10.066517] NET: Registered PF_PACKET protocol family gitlab # [ 9.814869] dhcpcd[1248]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.820266] dhcpcd[1248]: eth0: probing address 10.0.2.15/24 gitlab # [ 10.257525] postgresql-pre-start[1277]: performing post-bootstrap initialization ... ok gitlab # [ 10.635537] postgresql-pre-start[1277]: syncing data to disk ... ok gitlab # [ 10.639227] postgresql-pre-start[1277]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 10.641869] postgresql-pre-start[1277]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 10.646269] postgresql-pre-start[1277]: Success. You can now start the database server using: gitlab # [ 10.649265] postgresql-pre-start[1277]: pg_ctl -D /var/lib/postgresql/17 -l logfile start gitlab # [ 10.758706] postgres[1332]: [1332] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 10.765934] postgres[1332]: [1332] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 10.769264] postgres[1332]: [1332] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 10.778304] postgres[1332]: [1332] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 10.795362] postgres[1338]: [1338] LOG: database system was shut down at 2026-07-09 06:48:59 GMT gitlab # [ 10.805958] postgres[1332]: [1332] LOG: database system is ready to accept connections gitlab # [ 10.812801] systemd[1]: Started PostgreSQL Server. gitlab # [ 10.820631] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.998557] postgresql-setup-start[1358]: CREATE DATABASE gitlab # [ 11.039612] postgresql-setup-start[1368]: CREATE ROLE gitlab # [ 11.055492] postgresql-setup-start[1370]: ALTER DATABASE gitlab # [ 11.091199] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1363]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 11.096988] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1363]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 11.103331] postgresql-setup-start[1375]: CREATE ROLE gitlab # [ 11.106694] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 11.110219] systemd[1]: Reached target PostgreSQL. gitlab # [ 11.112745] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 11.142581] systemd[1]: Finished gitlab-config.service. gitlab # [ 11.147507] systemd[1]: Started gitaly.service. gitlab # [ 11.149514] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 11.213892] gitlab-postgresql-start[1394]: CREATE DATABASE gitlab # [ 11.260114] gitlab-postgresql-start[1404]: CREATE EXTENSION gitlab # [ 11.300507] gitlab-postgresql-start[1406]: CREATE EXTENSION gitlab # [ 11.304522] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 11.420504] dhcpcd[1248]: eth0: soliciting an IPv6 router gitlab # [ 11.423589] dhcpcd[1248]: eth0: Router Advertisement from fe80::2 gitlab # [ 11.425139] dhcpcd[1248]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 11.427105] dhcpcd[1248]: eth0: adding route to fec0::/64 gitlab # [ 11.428759] dhcpcd[1248]: eth0: adding default route via fe80::2 gitlab # [ 11.695457] gitaly[1397]: time="2026-07-09T06:49:00.494Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=1397 gitlab # [ 11.697627] gitaly[1397]: time="2026-07-09T06:49:00.494Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=1397 gitlab # [ 11.699537] gitaly[1397]: time="2026-07-09T06:49:00.494Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=1397 gitlab # [ 11.702420] gitaly[1397]: time="2026-07-09T06:49:00.494Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=1397 gitlab # [ 11.703950] gitaly[1397]: time="2026-07-09T06:49:00.495Z" level=info msg="Starting Gitaly" pid=1397 version=18.11.4 gitlab # [ 11.899498] gitaly[1397]: time="2026-07-09T06:49:00.702Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=207 pid=1397 gitlab # [ 11.901126] gitaly[1397]: time="2026-07-09T06:49:00.703Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=1397 gitlab # [ 11.901662] gitaly[1397]: time="2026-07-09T06:49:00.703Z" level=info msg="finished initializing command factory" duration_ms=0 pid=1397 gitlab # [ 11.902807] gitaly[1397]: time="2026-07-09T06:49:00.703Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1397/git-exec-1981981585.d/git pid=1397 gitlab # [ 11.906141] gitaly[1397]: time="2026-07-09T06:49:00.709Z" level=info msg="finished detecting git version" duration_ms=5 pid=1397 gitlab # [ 11.906990] gitaly[1397]: time="2026-07-09T06:49:00.709Z" level=info msg="using Git version" pid=1397 version=2.53.g gitlab # [ 11.907915] gitaly[1397]: time="2026-07-09T06:49:00.709Z" level=info msg="clearing disk cache object folder" pid=1397 storage=default gitlab # [ 11.908984] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4229726837 pid=1397 storage=default gitlab # [ 11.910196] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4229726837 pid=1397 storage=default gitlab # [ 11.910803] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=1397 gitlab # [ 11.911859] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=1397 gitlab # [ 11.913049] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4229726837 clear_duration_ms=0 pid=1397 storage=default gitlab # [ 11.977189] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="counting repositories" pid=1397 gitlab # [ 11.978839] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="starting to count repositories" pid=1397 storage_path=/var/gitlab/state/repositories gitlab # [ 11.980324] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1397 storage_path=/var/gitlab/state/repositories gitlab # [ 11.981966] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="completed counting all repositories" counting_duration=0 pid=1397 gitlab # [ 11.983331] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=1397 gitlab # [ 11.983991] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="finished tempdir cleaner walk" pid=1397 storage=default time_ms=0 gitlab # [ 11.985046] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1397 schema=unix gitlab # [ 11.986087] gitaly[1397]: time="2026-07-09T06:49:00.781Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1397/sock.d/intern pid=1397 schema=unix gitlab # [ 12.005133] gitaly[1397]: time="2026-07-09T06:49:00.808Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=1397 gitlab # [ 12.005988] gitaly[1397]: time="2026-07-09T06:49:00.808Z" level=info msg="Started Gitaly" duration_ms=313 pid=1397 gitlab # [ 12.011225] gitaly[1397]: time="2026-07-09T06:49:00.814Z" level=info msg="maintenance: daily scheduled" pid=1397 scheduled="2026-07-09 12:00:00 +0000 UTC" gitlab # [ 12.626644] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1400]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.631695] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1400]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.652763] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1400]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.880240] gitaly[1397]: time="2026-07-09T06:49:02.682Z" level=info msg="License database preloaded" duration_ms=2186 pid=1397 gitlab # [ 15.173582] dhcpcd[1248]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 15.177730] dhcpcd[1248]: eth0: adding route to 10.0.2.0/24 gitlab # [ 15.181161] dhcpcd[1248]: eth0: adding default route via 10.0.2.2 gitlab # [ 15.312960] systemd[1]: Started DHCP Client. gitlab # [ 15.318834] systemd[1]: Reached target Network is Online. gitlab # [ 44.007829] postgres[1770]: [1770] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 44.012554] postgres[1770]: [1770] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 44.015884] postgres[1770]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 44.021814] postgres[1770]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 44.026750] postgres[1770]: attidentity AS identity, gitlab # [ 44.030314] postgres[1770]: attgenerated as attgenerated gitlab # [ 44.034361] postgres[1770]: FROM pg_attribute a gitlab # [ 44.037565] postgres[1770]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 44.042159] postgres[1770]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 44.046283] postgres[1770]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 44.049718] postgres[1770]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 44.053514] postgres[1770]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 44.057101] postgres[1770]: ORDER BY a.attnum gitlab # [ 44.059475] postgres[1770]: gitlab # [ 44.242113] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1400]: W, [2026-07-09T06:49:33.044452 #1400] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 49.732770] gitlab-workhorse[1306]: time="2026-07-09T06:49:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 49.891736] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1830]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 49.895331] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1830]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 59.743426] gitlab-workhorse[1306]: time="2026-07-09T06:49:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 67.588863] postgres[1914]: [1914] ERROR: relation "table_without_models" does not exist gitlab # [ 67.590858] postgres[1914]: [1914] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 69.754119] gitlab-workhorse[1306]: time="2026-07-09T06:49:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.764102] gitlab-workhorse[1306]: time="2026-07-09T06:50:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 82.573767] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 82.577960] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 82.581662] systemd[1]: Starting gitlab.service... gitlab # [ 82.953780] bundle[2117]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 82.955871] bundle[2117]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 82.962163] bundle[2117]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 85.883623] sidekiq-cluster[2116]: {"severity":"INFO","time":"2026-07-09T06:50:14.668Z","message":"Starting cluster with 1 processes"} gitlab # [ 86.058518] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.839Z","pid":2117,"message":"* Enabling systemd notification integration"} gitlab # [ 86.061179] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"Puma starting in cluster mode..."} gitlab # [ 86.063884] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Puma version: 7.2.0 (\"On The Corner\")"} gitlab # [ 86.067318] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 86.069779] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Min threads: 0"} gitlab # [ 86.073154] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Max threads: 4"} gitlab # [ 86.076287] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Environment: production"} gitlab # [ 86.078927] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Master PID: 2117"} gitlab # [ 86.082948] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Workers: 2"} gitlab # [ 86.086988] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 86.089813] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Preloading application"} gitlab # [ 89.773671] gitlab-workhorse[1306]: time="2026-07-09T06:50:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 99.775588] gitlab-workhorse[1306]: time="2026-07-09T06:50:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.777316] gitlab-workhorse[1306]: time="2026-07-09T06:50:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 111.030267] bundle[2117]: W, [2026-07-09T06:50:39.832713 #2117] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 113.456834] sidekiq-cluster[2147]: 2026-07-09T06:50:42.259Z pid=2147 tid=u0b INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2147, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426627072} gitlab # [ 113.644758] sidekiq-cluster[2147]: W, [2026-07-09T06:50:42.447943 #2147] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 113.683975] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:42.487Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 115.714167] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.516Z","message":"GitLab reliable fetch activated!"} gitlab # [ 115.720633] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.516Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 115.743263] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.546Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 115.759239] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.562Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 115.768116] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.568Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 115.779922] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.583Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 115.793271] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.596Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 115.805160] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.608Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 115.817517] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.620Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 115.829417] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.632Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 115.841593] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.644Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"} gitlab # [ 115.854344] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.657Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 115.866564] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.669Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 115.878221] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.681Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 115.887178] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.690Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 115.896184] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.699Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 115.905238] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.708Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 115.915482] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.718Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"} gitlab # [ 115.924879] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.728Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 115.935530] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.738Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 115.945389] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.748Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 115.956788] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.760Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 115.968414] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.771Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 115.979832] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.783Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 115.990188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.793Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 116.000625] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.803Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 116.010180] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.813Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 116.019767] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.823Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 116.030367] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.833Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 116.040527] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.843Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 116.050806] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.854Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 116.061510] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.864Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 116.072178] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.875Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 116.081600] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.884Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 116.092639] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.895Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 116.102533] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.905Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 116.113141] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.916Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 116.121594] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.924Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 116.129802] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.933Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 116.137789] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.941Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 116.147221] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.950Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 116.155130] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.958Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 116.163143] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.966Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 116.171561] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.974Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 116.179928] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.983Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 116.191083] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.994Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 116.200398] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.003Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 116.248535] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.051Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 116.254490] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.057Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 116.261420] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.064Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 116.267606] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.071Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 116.274319] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.077Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 116.280742] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.084Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 116.286645] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.090Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 116.293166] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.096Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 116.306382] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.109Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 116.313394] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.116Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 116.323054] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.126Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 116.331092] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.134Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 116.339468] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.142Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 116.347662] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.151Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 116.355606] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.158Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 116.364575] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.167Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 116.372657] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.176Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 116.382275] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.185Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 116.391911] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.195Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 116.401393] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.204Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 116.410190] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.213Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 116.418744] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.222Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 116.426700] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.230Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 116.437182] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.240Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 116.445714] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.249Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 116.455205] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.258Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 116.463587] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.266Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 116.472582] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.275Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 116.481194] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.284Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 116.489939] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.293Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 116.499354] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.302Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 116.508369] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.311Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 116.517994] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.321Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 116.527207] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.330Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 116.536367] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.339Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 116.545191] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.348Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 116.553632] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.356Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 116.562605] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.365Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 116.573377] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.376Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 116.583570] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.386Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 116.594226] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.397Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 116.603176] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.406Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 116.613620] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.416Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 116.622334] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.425Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 116.630125] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.433Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 116.638137] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.441Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 116.645593] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.448Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 116.653201] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.456Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 116.661700] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.465Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 116.669216] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.472Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 116.677273] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.480Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 116.689891] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.493Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 116.699877] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.503Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 116.708651] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.511Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 116.737431] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.539Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 116.742127] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.545Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 116.750335] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.553Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 116.763558] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.566Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 116.774253] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.577Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 116.783390] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.586Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 116.793183] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.596Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 116.801951] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.605Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 116.811188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.614Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 116.821861] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.625Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 116.830620] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.633Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 119.787851] gitlab-workhorse[1306]: time="2026-07-09T06:50:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 129.791135] gitlab-workhorse[1306]: time="2026-07-09T06:50:58Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 137.793850] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.596Z","message":"Booted Rails 7.2.3 application in production environment"} gitlab # [ 137.797664] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.600Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 137.801048] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.600Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 137.803987] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.600Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 137.807363] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.603Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 139.171110] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:07.973Z","message":"Cleaning working queues"} gitlab # [ 139.179534] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:07.982Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b28303ae4f19620a80d00c18","created_at":"2026-07-09T06:50:10.713Z","trace_propagation_headers":{"sentry-trace":"abdbc09bb6024011b71866522215552f-29c5355ce1e2476f","baggage":"sentry-trace_id=abdbc09bb6024011b71866522215552f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"5a5c57f4ba6b5f96e50fe7e79de8e58f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-09T06:50:10.724Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b28303ae4f19620a80d00c18: start","job_status":"start","queue_duration_s":57.258307,"scheduling_latency_s":57.25837} gitlab # [ 139.430626] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:08.232Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b28303ae4f19620a80d00c18","created_at":"2026-07-09T06:50:10.713Z","trace_propagation_headers":{"sentry-trace":"abdbc09bb6024011b71866522215552f-29c5355ce1e2476f","baggage":"sentry-trace_id=abdbc09bb6024011b71866522215552f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"5a5c57f4ba6b5f96e50fe7e79de8e58f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-09T06:50:10.724Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b28303ae4f19620a80d00c18: done: 0.250506 sec","job_status":"done","queue_duration_s":57.258307,"scheduling_latency_s":57.25837,"redis_calls":30,"redis_duration_s":0.016214,"redis_read_bytes":410,"redis_write_bytes":3701,"redis_feature_flag_calls":19,"redis_feature_flag_duration_s":0.009991,"redis_feature_flag_read_bytes":400,"redis_feature_flag_write_bytes":2454,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004073,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00215,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.106646794,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.250506,"completed_at":"2026-07-09T06:51:08.232Z","load_balancing_strategy":"primary","db_duration_s":0.088455,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 139.456556] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 139.458248] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 139.460013] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"! #\u003cThread:0x00007678ab45dfb8 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 139.461365] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"! #\u003cThread:0x00007678ab6787f8 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 139.463130] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"Use Ctrl-C to stop"} gitlab # [ 139.464599] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.260Z","pid":2117,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 139.863263] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.666Z","pid":2117,"message":"- Worker 0 (PID: 2666) booted in 0.34s, phase: 0"} gitlab # [ 139.904638] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.707Z","pid":2117,"message":"- Worker 1 (PID: 2670) booted in 0.36s, phase: 0"} gitlab # [ 139.918220] systemd[1]: Started gitlab.service. gitlab # [ 139.923363] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 139.928452] systemd[1]: Starting setup-gitlab.service... gitlab # [ 139.946663] setup-gitlab-start[2706]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 142.292589] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:51:11 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 2299 gitlab # [ 142.300946] setup-gitlab-start[2726]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token gitlab # [ 142.310961] setup-gitlab-start[2727]: ++ jq -r .access_token gitlab # [ 143.104782] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:51:11 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 785 gitlab # [ 143.109971] setup-gitlab-start[2706]: + BEARER=1ada74272ecd2e25f1eb27e88b9ead10e8deb7835364224985b565ea2981f204 gitlab # [ 143.112326] setup-gitlab-start[2706]: + AUTH='Authorization: Bearer 1ada74272ecd2e25f1eb27e88b9ead10e8deb7835364224985b565ea2981f204' gitlab # [ 143.114870] setup-gitlab-start[2706]: + curl -fs -X PUT -H 'Authorization: Bearer 1ada74272ecd2e25f1eb27e88b9ead10e8deb7835364224985b565ea2981f204' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true' gitlab # [ 144.148437] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:51:12 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 1015 gitlab # [ 144.159720] setup-gitlab-start[2706]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear gitlab # [ 144.168236] runuser[2747]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 144.541358] setup-gitlab-start[2748]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 144.544176] setup-gitlab-start[2748]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 144.546291] setup-gitlab-start[2748]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 145.117970] setup-gitlab-start[2748]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 145.126726] setup-gitlab-start[2748]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 145.135221] setup-gitlab-start[2748]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 152.944985] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"70a7aea24063a326f42e21f4","created_at":"2026-07-09T06:51:21.725Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f4f9d22debb3ce13bf038e8f294fbf2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-09T06:51:21.743Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-70a7aea24063a326f42e21f4: start","job_status":"start","queue_duration_s":0.003035,"scheduling_latency_s":0.003105} gitlab # [ 153.113279] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.916Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"70a7aea24063a326f42e21f4","created_at":"2026-07-09T06:51:21.725Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f4f9d22debb3ce13bf038e8f294fbf2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-09T06:51:21.743Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-70a7aea24063a326f42e21f4: done: 0.169404 sec","job_status":"done","queue_duration_s":0.003035,"scheduling_latency_s":0.003105,"redis_calls":10,"redis_duration_s":0.027127000000000002,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001902,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.021034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004191,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003399874,"duration_s":0.169404,"completed_at":"2026-07-09T06:51:21.915Z","load_balancing_strategy":"primary","db_duration_s":0.018193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.203677] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.919Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8b0043a41667db9be91b7ce6","created_at":"2026-07-09T06:51:21.810Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97c6722523f6c61b8686cb0f4c25b8a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:21.846Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8b0043a41667db9be91b7ce6: start","job_status":"start","queue_duration_s":0.073002,"scheduling_latency_s":0.073042} gitlab # [ 153.226496] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.940Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 153.232077] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8b0043a41667db9be91b7ce6","created_at":"2026-07-09T06:51:21.810Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97c6722523f6c61b8686cb0f4c25b8a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:21.846Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8b0043a41667db9be91b7ce6: done: 0.02798 sec","job_status":"done","queue_duration_s":0.073002,"scheduling_latency_s":0.073042,"redis_calls":10,"redis_duration_s":0.004849,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000952,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000496432,"duration_s":0.02798,"completed_at":"2026-07-09T06:51:21.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000897,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.268635] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"16e3ec86d6c96d1ce2441143","created_at":"2026-07-09T06:51:21.890Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e9989e2223533b3d0742ef3f5e5f014","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.021Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-16e3ec86d6c96d1ce2441143: start","job_status":"start","queue_duration_s":0.017393,"scheduling_latency_s":0.017457} gitlab # [ 153.414608] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"16e3ec86d6c96d1ce2441143","created_at":"2026-07-09T06:51:21.890Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e9989e2223533b3d0742ef3f5e5f014","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.021Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-16e3ec86d6c96d1ce2441143: done: 0.178395 sec","job_status":"done","queue_duration_s":0.017393,"scheduling_latency_s":0.017457,"redis_calls":13,"redis_duration_s":0.089392,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.020907,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.009493,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.058992,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03048097,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.178395,"completed_at":"2026-07-09T06:51:22.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015872,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 153.505635] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4ad7defeb37b96ba168d54cd","created_at":"2026-07-09T06:51:22.038Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55ff693b59552cbcbeebbdecf622dadf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.047Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4ad7defeb37b96ba168d54cd: start","job_status":"start","queue_duration_s":0.261162,"scheduling_latency_s":0.261227} gitlab # [ 153.602534] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4ad7defeb37b96ba168d54cd","created_at":"2026-07-09T06:51:22.038Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55ff693b59552cbcbeebbdecf622dadf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.047Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4ad7defeb37b96ba168d54cd: done: 0.097427 sec","job_status":"done","queue_duration_s":0.261162,"scheduling_latency_s":0.261227,"redis_calls":12,"redis_duration_s":0.043005,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.028406,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011899,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000612089,"duration_s":0.097427,"completed_at":"2026-07-09T06:51:22.405Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.652415] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5f51e811fd3cb041cf5391d8","created_at":"2026-07-09T06:51:22.064Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d942e171022ae1869fb5b03b95f4196","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.125Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5f51e811fd3cb041cf5391d8: start","job_status":"start","queue_duration_s":0.286806,"scheduling_latency_s":0.286867} gitlab # [ 153.664697] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5f51e811fd3cb041cf5391d8","created_at":"2026-07-09T06:51:22.064Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d942e171022ae1869fb5b03b95f4196","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.125Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5f51e811fd3cb041cf5391d8: done: 0.055716 sec","job_status":"done","queue_duration_s":0.286806,"scheduling_latency_s":0.286867,"redis_calls":12,"redis_duration_s":0.003893,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000715,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001125004,"exclusive_lock_hold_duration_s":0.005300394,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.055716,"completed_at":"2026-07-09T06:51:22.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.734275] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2cad8cfdb593de9f7688bfed","created_at":"2026-07-09T06:51:22.162Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65186eed3a203f686f4317392c151bab","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.180Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2cad8cfdb593de9f7688bfed: start","job_status":"start","queue_duration_s":0.289366,"scheduling_latency_s":0.289386} gitlab # [ 154.234272] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2cad8cfdb593de9f7688bfed","created_at":"2026-07-09T06:51:22.162Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65186eed3a203f686f4317392c151bab","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.180Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2cad8cfdb593de9f7688bfed: done: 0.567592 sec","job_status":"done","queue_duration_s":0.289366,"scheduling_latency_s":0.289386,"redis_calls":574,"redis_duration_s":0.24959599999999998,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.247695,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.357253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000358426,"duration_s":0.567592,"completed_at":"2026-07-09T06:51:23.036Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.339195] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.142Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cfb31c9f7612c04a003c09e1","created_at":"2026-07-09T06:51:22.201Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b3e095e33be9fe5a140c3761a0f9ab0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.220Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cfb31c9f7612c04a003c09e1: start","job_status":"start","queue_duration_s":0.922182,"scheduling_latency_s":0.922219} gitlab # [ 154.401558] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cfb31c9f7612c04a003c09e1","created_at":"2026-07-09T06:51:22.201Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b3e095e33be9fe5a140c3761a0f9ab0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.220Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cfb31c9f7612c04a003c09e1: done: 0.062114 sec","job_status":"done","queue_duration_s":0.922182,"scheduling_latency_s":0.922219,"redis_calls":12,"redis_duration_s":0.004444,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000615,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002449,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000996495,"exclusive_lock_hold_duration_s":0.011014528,"duration_s":0.062114,"completed_at":"2026-07-09T06:51:23.204Z","load_balancing_strategy":"primary","db_duration_s":0.005412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.438081] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e8edd68e2f315f9fabbe4d8c","created_at":"2026-07-09T06:51:22.319Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9c84e5c3bfb13a87145eafa37340e4e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-09T06:51:22.345Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e8edd68e2f315f9fabbe4d8c: start","job_status":"start","queue_duration_s":0.862396,"scheduling_latency_s":0.862445} gitlab # [ 154.439242] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.231Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 154.440092] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e8edd68e2f315f9fabbe4d8c","created_at":"2026-07-09T06:51:22.319Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9c84e5c3bfb13a87145eafa37340e4e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-09T06:51:22.345Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e8edd68e2f315f9fabbe4d8c: done: 0.031173 sec","job_status":"done","queue_duration_s":0.862396,"scheduling_latency_s":0.862445,"redis_calls":10,"redis_duration_s":0.005026,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001199,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000600914,"duration_s":0.031173,"completed_at":"2026-07-09T06:51:23.238Z","load_balancing_strategy":"primary","db_duration_s":0.000935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.555668] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8d6fd29b3ff90f7ac5a8a8a9","created_at":"2026-07-09T06:51:22.386Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.399Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8d6fd29b3ff90f7ac5a8a8a9: start","job_status":"start","queue_duration_s":0.959363,"scheduling_latency_s":0.959401} gitlab # [ 154.719932] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8d6fd29b3ff90f7ac5a8a8a9","created_at":"2026-07-09T06:51:22.386Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.399Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8d6fd29b3ff90f7ac5a8a8a9: done: 0.164311 sec","job_status":"done","queue_duration_s":0.959363,"scheduling_latency_s":0.959401,"redis_calls":71,"redis_duration_s":0.034127,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004633,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001596,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.02658,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001318,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.104554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000667683,"duration_s":0.164311,"completed_at":"2026-07-09T06:51:23.522Z","load_balancing_strategy":"primary","db_duration_s":0.003897,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.825459] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3625be359b6b2aebb3dd8958","created_at":"2026-07-09T06:51:22.411Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da756f5492c3f47b127605fda5e474ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3625be359b6b2aebb3dd8958: start","job_status":"start","queue_duration_s":1.029522,"scheduling_latency_s":1.029585} gitlab # [ 155.272795] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3625be359b6b2aebb3dd8958","created_at":"2026-07-09T06:51:22.411Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da756f5492c3f47b127605fda5e474ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3625be359b6b2aebb3dd8958: done: 0.446971 sec","job_status":"done","queue_duration_s":1.029522,"scheduling_latency_s":1.029585,"redis_calls":14,"redis_duration_s":0.008429,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002559,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002373,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.055,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.340636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001403251,"exclusive_lock_hold_duration_s":0.40073082000000004,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.446971,"completed_at":"2026-07-09T06:51:24.074Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.095728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.370689] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"089f7e50bd695d7739f1be5f","created_at":"2026-07-09T06:51:22.606Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9f865d7ee5cef95c619d9d51cc488fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.615Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-089f7e50bd695d7739f1be5f: start","job_status":"start","queue_duration_s":1.558499,"scheduling_latency_s":1.558539} gitlab # [ 155.451449] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"089f7e50bd695d7739f1be5f","created_at":"2026-07-09T06:51:22.606Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9f865d7ee5cef95c619d9d51cc488fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.615Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-089f7e50bd695d7739f1be5f: done: 0.080337 sec","job_status":"done","queue_duration_s":1.558499,"scheduling_latency_s":1.558539,"redis_calls":12,"redis_duration_s":0.0060609999999999995,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000845,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002257,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000775238,"exclusive_lock_hold_duration_s":0.028949769,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.080337,"completed_at":"2026-07-09T06:51:24.253Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.535385] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"91b99ca81f7852a09c971f55","created_at":"2026-07-09T06:51:22.630Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c345a84abfbbb1d3dc8159fd7b32a94a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.639Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-91b99ca81f7852a09c971f55: start","job_status":"start","queue_duration_s":1.698439,"scheduling_latency_s":1.698504} gitlab # [ 155.581304] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.384Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 155.599497] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"91b99ca81f7852a09c971f55","created_at":"2026-07-09T06:51:22.630Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c345a84abfbbb1d3dc8159fd7b32a94a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.639Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-91b99ca81f7852a09c971f55: done: 0.064371 sec","job_status":"done","queue_duration_s":1.698439,"scheduling_latency_s":1.698504,"redis_calls":10,"redis_duration_s":0.007035,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000869,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005431,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000682768,"duration_s":0.064371,"completed_at":"2026-07-09T06:51:24.402Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001303,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.668376] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.471Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1216b3e01fbd020c0d7834e6","created_at":"2026-07-09T06:51:22.650Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d38eac13f6572bdf71eaee4a87d4f1b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.657Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1216b3e01fbd020c0d7834e6: start","job_status":"start","queue_duration_s":1.813662,"scheduling_latency_s":1.813724} gitlab # [ 155.715225] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1216b3e01fbd020c0d7834e6","created_at":"2026-07-09T06:51:22.650Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d38eac13f6572bdf71eaee4a87d4f1b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.657Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1216b3e01fbd020c0d7834e6: done: 0.046539 sec","job_status":"done","queue_duration_s":1.813662,"scheduling_latency_s":1.813724,"redis_calls":10,"redis_duration_s":0.005738,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001117,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000940064,"duration_s":0.046539,"completed_at":"2026-07-09T06:51:24.517Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001176,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.785288] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6680868bb36c4c1f7998acc9","created_at":"2026-07-09T06:51:22.665Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"049cff5f35e1590950627fcbadc149d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.671Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6680868bb36c4c1f7998acc9: start","job_status":"start","queue_duration_s":1.916542,"scheduling_latency_s":1.916607} gitlab # [ 155.814556] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.617Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 155.830203] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6680868bb36c4c1f7998acc9","created_at":"2026-07-09T06:51:22.665Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"049cff5f35e1590950627fcbadc149d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.671Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6680868bb36c4c1f7998acc9: done: 0.045028 sec","job_status":"done","queue_duration_s":1.916542,"scheduling_latency_s":1.916607,"redis_calls":10,"redis_duration_s":0.006300999999999999,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001316,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000922185,"duration_s":0.045028,"completed_at":"2026-07-09T06:51:24.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.833497] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a86cba595c3feb0447bae7dd","created_at":"2026-07-09T06:51:22.678Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7470f8bd5c246cddd215dbc2b8bc6b93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-09T06:51:22.685Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a86cba595c3feb0447bae7dd: start","job_status":"start","queue_duration_s":1.951055,"scheduling_latency_s":1.951094} gitlab # [ 155.924982] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.728Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 155.938954] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.741Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a86cba595c3feb0447bae7dd","created_at":"2026-07-09T06:51:22.678Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7470f8bd5c246cddd215dbc2b8bc6b93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-09T06:51:22.685Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a86cba595c3feb0447bae7dd: done: 0.104756 sec","job_status":"done","queue_duration_s":1.951055,"scheduling_latency_s":1.951094,"redis_calls":10,"redis_duration_s":0.004847,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001192,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.072,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.072,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000948165,"duration_s":0.104756,"completed_at":"2026-07-09T06:51:24.741Z","load_balancing_strategy":"primary","db_duration_s":0.072147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.006130] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.808Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"71ef612bb6e72fa4f7865a67","created_at":"2026-07-09T06:51:22.699Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"569a577e56c5f49c21124b85e6c2684a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.712Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-71ef612bb6e72fa4f7865a67: start","job_status":"start","queue_duration_s":2.09672,"scheduling_latency_s":2.096783} gitlab # [ 156.108617] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.911Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"71ef612bb6e72fa4f7865a67","created_at":"2026-07-09T06:51:22.699Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"569a577e56c5f49c21124b85e6c2684a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.712Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-71ef612bb6e72fa4f7865a67: done: 0.102641 sec","job_status":"done","queue_duration_s":2.09672,"scheduling_latency_s":2.096783,"redis_calls":16,"redis_duration_s":0.008469,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.004132,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000997892,"duration_s":0.102641,"completed_at":"2026-07-09T06:51:24.911Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.113136] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.915Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bcdfcb94d118b8031640a1cb","created_at":"2026-07-09T06:51:22.722Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.731Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bcdfcb94d118b8031640a1cb: start","job_status":"start","queue_duration_s":2.183705,"scheduling_latency_s":2.183757} gitlab # [ 156.213486] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bcdfcb94d118b8031640a1cb","created_at":"2026-07-09T06:51:22.722Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.731Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bcdfcb94d118b8031640a1cb: done: 0.039238 sec","job_status":"done","queue_duration_s":2.183705,"scheduling_latency_s":2.183757,"redis_calls":19,"redis_duration_s":0.005241,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001051,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000399,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002675,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000439441,"duration_s":0.039238,"completed_at":"2026-07-09T06:51:24.954Z","load_balancing_strategy":"primary","db_duration_s":0.00469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.218968] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d6e9e02254cfdfe14c32b3a1","created_at":"2026-07-09T06:51:22.740Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"404265cf8c38963254e52bb15fcb58ae","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.752Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d6e9e02254cfdfe14c32b3a1: start","job_status":"start","queue_duration_s":2.2037,"scheduling_latency_s":2.20372} gitlab # [ 156.235602] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d6e9e02254cfdfe14c32b3a1","created_at":"2026-07-09T06:51:22.740Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"404265cf8c38963254e52bb15fcb58ae","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.752Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d6e9e02254cfdfe14c32b3a1: done: 0.022765 sec","job_status":"done","queue_duration_s":2.2037,"scheduling_latency_s":2.20372,"redis_calls":10,"redis_duration_s":0.003487,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000701,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002049,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000737,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000484978,"duration_s":0.022765,"completed_at":"2026-07-09T06:51:24.978Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.315522] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ba281a9fb713ff50542480af","created_at":"2026-07-09T06:51:22.767Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01822331cc054980c8ce36f766af0a16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.776Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ba281a9fb713ff50542480af: start","job_status":"start","queue_duration_s":2.204081,"scheduling_latency_s":2.204101} gitlab # [ 156.321369] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.992Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 156.392596] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ba281a9fb713ff50542480af","created_at":"2026-07-09T06:51:22.767Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01822331cc054980c8ce36f766af0a16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.776Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ba281a9fb713ff50542480af: done: 0.018756 sec","job_status":"done","queue_duration_s":2.204081,"scheduling_latency_s":2.204101,"redis_calls":10,"redis_duration_s":0.003618,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000751,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000486933,"duration_s":0.018756,"completed_at":"2026-07-09T06:51:24.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.394874] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4c4ee8f1738e8dcc045f043a","created_at":"2026-07-09T06:51:22.785Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fde370cecc046592443ddc5e7dc7660e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.796Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4c4ee8f1738e8dcc045f043a: start","job_status":"start","queue_duration_s":2.205152,"scheduling_latency_s":2.205172} gitlab # [ 156.464286] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.048Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4c4ee8f1738e8dcc045f043a","created_at":"2026-07-09T06:51:22.785Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fde370cecc046592443ddc5e7dc7660e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.796Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4c4ee8f1738e8dcc045f043a: done: 0.046923 sec","job_status":"done","queue_duration_s":2.205152,"scheduling_latency_s":2.205172,"redis_calls":12,"redis_duration_s":0.004666,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000872,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002555,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057214,"exclusive_lock_hold_duration_s":0.016844319,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.046923,"completed_at":"2026-07-09T06:51:25.048Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.483891] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e68726b229cf9d3598ee85a1","created_at":"2026-07-09T06:51:22.808Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2ba6187a45160e2eb4922b243fa9de7","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.819Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e68726b229cf9d3598ee85a1: start","job_status":"start","queue_duration_s":2.230288,"scheduling_latency_s":2.230308} gitlab # [ 156.553280] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e68726b229cf9d3598ee85a1","created_at":"2026-07-09T06:51:22.808Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2ba6187a45160e2eb4922b243fa9de7","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.819Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e68726b229cf9d3598ee85a1: done: 0.093505 sec","job_status":"done","queue_duration_s":2.230288,"scheduling_latency_s":2.230308,"redis_calls":10,"redis_duration_s":0.004342,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000994,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000988114,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.093505,"completed_at":"2026-07-09T06:51:25.143Z","load_balancing_strategy":"primary","db_duration_s":0.021208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.624921] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a452b5508fb5325765b9cc8a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a452b5508fb5325765b9cc8a: start","job_status":"start","queue_duration_s":1.632244,"scheduling_latency_s":1.632265} gitlab # [ 156.626339] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a452b5508fb5325765b9cc8a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a452b5508fb5325765b9cc8a: done: 0.026399 sec","job_status":"done","queue_duration_s":1.632244,"scheduling_latency_s":1.632265,"redis_calls":16,"redis_duration_s":0.004275,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0022,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000716851,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026399,"completed_at":"2026-07-09T06:51:25.171Z","load_balancing_strategy":"primary","db_duration_s":0.000629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.698750] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9b65a1edfd91a32d99e0629","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9b65a1edfd91a32d99e0629: start","job_status":"start","queue_duration_s":1.660597,"scheduling_latency_s":1.660619} gitlab # [ 156.700188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9b65a1edfd91a32d99e0629","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9b65a1edfd91a32d99e0629: done: 0.015714 sec","job_status":"done","queue_duration_s":1.660597,"scheduling_latency_s":1.660619,"redis_calls":13,"redis_duration_s":0.002729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001831,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015714,"completed_at":"2026-07-09T06:51:25.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.771852] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ef833a0b5fb0ad207b4443d","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ef833a0b5fb0ad207b4443d: start","job_status":"start","queue_duration_s":1.677581,"scheduling_latency_s":1.677601} gitlab # [ 156.773194] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ef833a0b5fb0ad207b4443d","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ef833a0b5fb0ad207b4443d: done: 0.014916 sec","job_status":"done","queue_duration_s":1.677581,"scheduling_latency_s":1.677601,"redis_calls":13,"redis_duration_s":0.003069,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001039,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014916,"completed_at":"2026-07-09T06:51:25.205Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.843818] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61bab904e1972e018f3234ff","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61bab904e1972e018f3234ff: start","job_status":"start","queue_duration_s":1.694039,"scheduling_latency_s":1.694059} gitlab # [ 156.845200] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61bab904e1972e018f3234ff","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61bab904e1972e018f3234ff: done: 0.014932 sec","job_status":"done","queue_duration_s":1.694039,"scheduling_latency_s":1.694059,"redis_calls":13,"redis_duration_s":0.0031060000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014932,"completed_at":"2026-07-09T06:51:25.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.937966] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e7f9159d150af9e2d9ba2b","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e7f9159d150af9e2d9ba2b: start","job_status":"start","queue_duration_s":1.710317,"scheduling_latency_s":1.710337} gitlab # [ 156.939053] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e7f9159d150af9e2d9ba2b","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e7f9159d150af9e2d9ba2b: done: 0.015735 sec","job_status":"done","queue_duration_s":1.710317,"scheduling_latency_s":1.710337,"redis_calls":13,"redis_duration_s":0.003664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002694,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015735,"completed_at":"2026-07-09T06:51:25.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.018160] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4febc853da917a8b0af44316","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4febc853da917a8b0af44316: start","job_status":"start","queue_duration_s":1.727319,"scheduling_latency_s":1.727339} gitlab # [ 157.019512] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4febc853da917a8b0af44316","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4febc853da917a8b0af44316: done: 0.015117 sec","job_status":"done","queue_duration_s":1.727319,"scheduling_latency_s":1.727339,"redis_calls":13,"redis_duration_s":0.00277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015117,"completed_at":"2026-07-09T06:51:25.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.090200] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac2dc358859fe81e0cea03e9","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac2dc358859fe81e0cea03e9: start","job_status":"start","queue_duration_s":1.743725,"scheduling_latency_s":1.743745} gitlab # [ 157.091780] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac2dc358859fe81e0cea03e9","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac2dc358859fe81e0cea03e9: done: 0.014978 sec","job_status":"done","queue_duration_s":1.743725,"scheduling_latency_s":1.743745,"redis_calls":13,"redis_duration_s":0.002963,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014978,"completed_at":"2026-07-09T06:51:25.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.165271] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"778c62d0a1c144615bc63a18","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-778c62d0a1c144615bc63a18: start","job_status":"start","queue_duration_s":1.760196,"scheduling_latency_s":1.76022} gitlab # [ 157.165972] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"778c62d0a1c144615bc63a18","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-778c62d0a1c144615bc63a18: done: 0.015225 sec","job_status":"done","queue_duration_s":1.760196,"scheduling_latency_s":1.76022,"redis_calls":13,"redis_duration_s":0.003043,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015225,"completed_at":"2026-07-09T06:51:25.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.249817] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd9276541a5766f9ae2b774f","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd9276541a5766f9ae2b774f: start","job_status":"start","queue_duration_s":1.77652,"scheduling_latency_s":1.77654} gitlab # [ 157.251195] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd9276541a5766f9ae2b774f","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd9276541a5766f9ae2b774f: done: 0.014782 sec","job_status":"done","queue_duration_s":1.77652,"scheduling_latency_s":1.77654,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014782,"completed_at":"2026-07-09T06:51:25.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.321286] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fb91c341766f35a48b9a41e","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fb91c341766f35a48b9a41e: start","job_status":"start","queue_duration_s":1.792557,"scheduling_latency_s":1.792577} gitlab # [ 157.322826] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fb91c341766f35a48b9a41e","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fb91c341766f35a48b9a41e: done: 0.014926 sec","job_status":"done","queue_duration_s":1.792557,"scheduling_latency_s":1.792577,"redis_calls":13,"redis_duration_s":0.003015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014926,"completed_at":"2026-07-09T06:51:25.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.418278] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bdedc38d17382c12b87bc56","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bdedc38d17382c12b87bc56: start","job_status":"start","queue_duration_s":1.808758,"scheduling_latency_s":1.808777} gitlab # [ 157.419391] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bdedc38d17382c12b87bc56","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bdedc38d17382c12b87bc56: done: 0.014953 sec","job_status":"done","queue_duration_s":1.808758,"scheduling_latency_s":1.808777,"redis_calls":13,"redis_duration_s":0.0027949999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014953,"completed_at":"2026-07-09T06:51:25.336Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.486424] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"139d66e0154b91f7ea8d4fb4","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-139d66e0154b91f7ea8d4fb4: start","job_status":"start","queue_duration_s":1.824962,"scheduling_latency_s":1.824983} gitlab # [ 157.487687] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"139d66e0154b91f7ea8d4fb4","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-139d66e0154b91f7ea8d4fb4: done: 0.015078 sec","job_status":"done","queue_duration_s":1.824962,"scheduling_latency_s":1.824983,"redis_calls":13,"redis_duration_s":0.002727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015078,"completed_at":"2026-07-09T06:51:25.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.561093] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e75d8f5804797e63b05034","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e75d8f5804797e63b05034: start","job_status":"start","queue_duration_s":1.841585,"scheduling_latency_s":1.841605} gitlab # [ 157.562393] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e75d8f5804797e63b05034","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e75d8f5804797e63b05034: done: 0.014915 sec","job_status":"done","queue_duration_s":1.841585,"scheduling_latency_s":1.841605,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014915,"completed_at":"2026-07-09T06:51:25.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.631983] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d75236282d60a2d8b0c12467","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d75236282d60a2d8b0c12467: start","job_status":"start","queue_duration_s":1.857762,"scheduling_latency_s":1.857781} gitlab # [ 157.633179] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d75236282d60a2d8b0c12467","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d75236282d60a2d8b0c12467: done: 0.015301 sec","job_status":"done","queue_duration_s":1.857762,"scheduling_latency_s":1.857781,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015301,"completed_at":"2026-07-09T06:51:25.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.731096] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92923f737911224179dfcd0d","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92923f737911224179dfcd0d: start","job_status":"start","queue_duration_s":1.874293,"scheduling_latency_s":1.874313} gitlab # [ 157.732386] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92923f737911224179dfcd0d","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92923f737911224179dfcd0d: done: 0.01498 sec","job_status":"done","queue_duration_s":1.874293,"scheduling_latency_s":1.874313,"redis_calls":13,"redis_duration_s":0.0027919999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-07-09T06:51:25.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.809235] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65ba9341a481e9321d87b3b6","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65ba9341a481e9321d87b3b6: start","job_status":"start","queue_duration_s":1.890553,"scheduling_latency_s":1.890572} gitlab # [ 157.810299] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65ba9341a481e9321d87b3b6","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65ba9341a481e9321d87b3b6: done: 0.015209 sec","job_status":"done","queue_duration_s":1.890553,"scheduling_latency_s":1.890572,"redis_calls":13,"redis_duration_s":0.003028,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015209,"completed_at":"2026-07-09T06:51:25.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.885271] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a98fab433ac2bc8bee18dfb9","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a98fab433ac2bc8bee18dfb9: start","job_status":"start","queue_duration_s":1.907387,"scheduling_latency_s":1.907408} gitlab # [ 157.886879] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a98fab433ac2bc8bee18dfb9","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a98fab433ac2bc8bee18dfb9: done: 0.01556 sec","job_status":"done","queue_duration_s":1.907387,"scheduling_latency_s":1.907408,"redis_calls":13,"redis_duration_s":0.003041,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01556,"completed_at":"2026-07-09T06:51:25.436Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.976399] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e31179a52e9ef3f551b70feb","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e31179a52e9ef3f551b70feb: start","job_status":"start","queue_duration_s":1.92428,"scheduling_latency_s":1.9243} gitlab # [ 157.978261] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e31179a52e9ef3f551b70feb","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e31179a52e9ef3f551b70feb: done: 0.014976 sec","job_status":"done","queue_duration_s":1.92428,"scheduling_latency_s":1.9243,"redis_calls":13,"redis_duration_s":0.002792,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014976,"completed_at":"2026-07-09T06:51:25.452Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.045993] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a97498bb491a8ef4cdc2748","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a97498bb491a8ef4cdc2748: start","job_status":"start","queue_duration_s":1.940558,"scheduling_latency_s":1.940579} gitlab # [ 158.047262] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a97498bb491a8ef4cdc2748","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a97498bb491a8ef4cdc2748: done: 0.014885 sec","job_status":"done","queue_duration_s":1.940558,"scheduling_latency_s":1.940579,"redis_calls":13,"redis_duration_s":0.0027790000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014885,"completed_at":"2026-07-09T06:51:25.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.117376] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8518462eb71ded0fd5dd82c4","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8518462eb71ded0fd5dd82c4: start","job_status":"start","queue_duration_s":1.956731,"scheduling_latency_s":1.956751} gitlab # [ 158.118966] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8518462eb71ded0fd5dd82c4","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8518462eb71ded0fd5dd82c4: done: 0.015424 sec","job_status":"done","queue_duration_s":1.956731,"scheduling_latency_s":1.956751,"redis_calls":13,"redis_duration_s":0.0027930000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015424,"completed_at":"2026-07-09T06:51:25.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.189764] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc5a9c2f1f76dde782f205c1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc5a9c2f1f76dde782f205c1: start","job_status":"start","queue_duration_s":1.973399,"scheduling_latency_s":1.973419} gitlab # [ 158.191193] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc5a9c2f1f76dde782f205c1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc5a9c2f1f76dde782f205c1: done: 0.015361 sec","job_status":"done","queue_duration_s":1.973399,"scheduling_latency_s":1.973419,"redis_calls":13,"redis_duration_s":0.0027140000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015361,"completed_at":"2026-07-09T06:51:25.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.287269] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d7de8028e9e2c439349d314","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d7de8028e9e2c439349d314: start","job_status":"start","queue_duration_s":1.990112,"scheduling_latency_s":1.990133} gitlab # [ 158.288843] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d7de8028e9e2c439349d314","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d7de8028e9e2c439349d314: done: 0.015702 sec","job_status":"done","queue_duration_s":1.990112,"scheduling_latency_s":1.990133,"redis_calls":13,"redis_duration_s":0.002993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015702,"completed_at":"2026-07-09T06:51:25.518Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.308256] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0eeebacaebdca8f92885f0a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0eeebacaebdca8f92885f0a: start","job_status":"start","queue_duration_s":2.00706,"scheduling_latency_s":2.00708} gitlab # [ 158.378708] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0eeebacaebdca8f92885f0a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0eeebacaebdca8f92885f0a: done: 0.015168 sec","job_status":"done","queue_duration_s":2.00706,"scheduling_latency_s":2.00708,"redis_calls":13,"redis_duration_s":0.002818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015168,"completed_at":"2026-07-09T06:51:25.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.386731] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06a1fa71cbf1bb73341667d8","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06a1fa71cbf1bb73341667d8: start","job_status":"start","queue_duration_s":2.023898,"scheduling_latency_s":2.023918} gitlab # [ 158.457630] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06a1fa71cbf1bb73341667d8","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06a1fa71cbf1bb73341667d8: done: 0.018317 sec","job_status":"done","queue_duration_s":2.023898,"scheduling_latency_s":2.023918,"redis_calls":13,"redis_duration_s":0.0031810000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018317,"completed_at":"2026-07-09T06:51:25.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.465413] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a3dfc692afe8a89994140b","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a3dfc692afe8a89994140b: start","job_status":"start","queue_duration_s":2.043804,"scheduling_latency_s":2.04383} gitlab # [ 158.538355] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a3dfc692afe8a89994140b","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a3dfc692afe8a89994140b: done: 0.015934 sec","job_status":"done","queue_duration_s":2.043804,"scheduling_latency_s":2.04383,"redis_calls":13,"redis_duration_s":0.002703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015934,"completed_at":"2026-07-09T06:51:25.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.545077] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec64d77e029bfc782237a852","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec64d77e029bfc782237a852: start","job_status":"start","queue_duration_s":2.061262,"scheduling_latency_s":2.061289} gitlab # [ 158.614520] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec64d77e029bfc782237a852","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec64d77e029bfc782237a852: done: 0.016831 sec","job_status":"done","queue_duration_s":2.061262,"scheduling_latency_s":2.061289,"redis_calls":13,"redis_duration_s":0.002853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016831,"completed_at":"2026-07-09T06:51:25.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.682482] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d93ab61c9b48b89a865fe064","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d93ab61c9b48b89a865fe064: start","job_status":"start","queue_duration_s":2.079355,"scheduling_latency_s":2.079385} gitlab # [ 158.683708] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d93ab61c9b48b89a865fe064","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d93ab61c9b48b89a865fe064: done: 0.015032 sec","job_status":"done","queue_duration_s":2.079355,"scheduling_latency_s":2.079385,"redis_calls":13,"redis_duration_s":0.002716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015032,"completed_at":"2026-07-09T06:51:25.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.702938] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81ee21e1764801bb0da3966a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81ee21e1764801bb0da3966a: start","job_status":"start","queue_duration_s":2.096069,"scheduling_latency_s":2.09609} gitlab # [ 158.773386] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81ee21e1764801bb0da3966a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81ee21e1764801bb0da3966a: done: 0.01498 sec","job_status":"done","queue_duration_s":2.096069,"scheduling_latency_s":2.09609,"redis_calls":13,"redis_duration_s":0.002771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-07-09T06:51:25.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.844110] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cd271a1b9315d33cab427a8","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cd271a1b9315d33cab427a8: start","job_status":"start","queue_duration_s":2.11226,"scheduling_latency_s":2.11228} gitlab # [ 158.845413] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cd271a1b9315d33cab427a8","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cd271a1b9315d33cab427a8: done: 0.014936 sec","job_status":"done","queue_duration_s":2.11226,"scheduling_latency_s":2.11228,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014936,"completed_at":"2026-07-09T06:51:25.640Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.914199] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"430f7a73b18aefd4c2d9b585","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-430f7a73b18aefd4c2d9b585: start","job_status":"start","queue_duration_s":2.128437,"scheduling_latency_s":2.128457} gitlab # [ 158.915802] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"430f7a73b18aefd4c2d9b585","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-430f7a73b18aefd4c2d9b585: done: 0.015221 sec","job_status":"done","queue_duration_s":2.128437,"scheduling_latency_s":2.128457,"redis_calls":13,"redis_duration_s":0.002996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015221,"completed_at":"2026-07-09T06:51:25.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.985252] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67d6e4b4dcc14be0b006e074","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67d6e4b4dcc14be0b006e074: start","job_status":"start","queue_duration_s":2.145044,"scheduling_latency_s":2.145063} gitlab # [ 158.986798] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67d6e4b4dcc14be0b006e074","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67d6e4b4dcc14be0b006e074: done: 0.015682 sec","job_status":"done","queue_duration_s":2.145044,"scheduling_latency_s":2.145063,"redis_calls":13,"redis_duration_s":0.0029890000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015682,"completed_at":"2026-07-09T06:51:25.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.061206] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30a8876d41e27c2ddb88784f","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30a8876d41e27c2ddb88784f: start","job_status":"start","queue_duration_s":2.162145,"scheduling_latency_s":2.162165} gitlab # [ 159.062884] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30a8876d41e27c2ddb88784f","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30a8876d41e27c2ddb88784f: done: 0.015349 sec","job_status":"done","queue_duration_s":2.162145,"scheduling_latency_s":2.162165,"redis_calls":13,"redis_duration_s":0.002902,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015349,"completed_at":"2026-07-09T06:51:25.690Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.160319] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60bd15f4ca420c51f31241ff","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60bd15f4ca420c51f31241ff: start","job_status":"start","queue_duration_s":2.178761,"scheduling_latency_s":2.178781} gitlab # [ 159.161889] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60bd15f4ca420c51f31241ff","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60bd15f4ca420c51f31241ff: done: 0.229893 sec","job_status":"done","queue_duration_s":2.178761,"scheduling_latency_s":2.178781,"redis_calls":13,"redis_duration_s":0.0031409999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.229893,"completed_at":"2026-07-09T06:51:25.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.235865] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15d4d88d092aab04ba0d61c1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15d4d88d092aab04ba0d61c1: start","job_status":"start","queue_duration_s":2.410255,"scheduling_latency_s":2.410282} gitlab # [ 159.237251] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15d4d88d092aab04ba0d61c1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15d4d88d092aab04ba0d61c1: done: 0.016505 sec","job_status":"done","queue_duration_s":2.410255,"scheduling_latency_s":2.410282,"redis_calls":13,"redis_duration_s":0.00273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016505,"completed_at":"2026-07-09T06:51:25.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.306764] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36f3d3419b51a3f829636dc5","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36f3d3419b51a3f829636dc5: start","job_status":"start","queue_duration_s":2.428066,"scheduling_latency_s":2.428086} gitlab # [ 159.308142] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36f3d3419b51a3f829636dc5","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36f3d3419b51a3f829636dc5: done: 0.014907 sec","job_status":"done","queue_duration_s":2.428066,"scheduling_latency_s":2.428086,"redis_calls":13,"redis_duration_s":0.00268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014907,"completed_at":"2026-07-09T06:51:25.956Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.385488] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e441b4aa605a6758af3eade","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e441b4aa605a6758af3eade: start","job_status":"start","queue_duration_s":2.44422,"scheduling_latency_s":2.44424} gitlab # [ 159.386979] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e441b4aa605a6758af3eade","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e441b4aa605a6758af3eade: done: 0.015078 sec","job_status":"done","queue_duration_s":2.44422,"scheduling_latency_s":2.44424,"redis_calls":13,"redis_duration_s":0.002966,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015078,"completed_at":"2026-07-09T06:51:25.972Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.458248] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec7523cc93b0ae16b72ed73","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec7523cc93b0ae16b72ed73: start","job_status":"start","queue_duration_s":2.460633,"scheduling_latency_s":2.460654} gitlab # [ 159.459386] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec7523cc93b0ae16b72ed73","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec7523cc93b0ae16b72ed73: done: 0.01517 sec","job_status":"done","queue_duration_s":2.460633,"scheduling_latency_s":2.460654,"redis_calls":13,"redis_duration_s":0.0028450000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01517,"completed_at":"2026-07-09T06:51:25.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.538225] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96a7887f853526ccf965a016","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96a7887f853526ccf965a016: start","job_status":"start","queue_duration_s":2.477047,"scheduling_latency_s":2.477067} gitlab # [ 159.539761] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96a7887f853526ccf965a016","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96a7887f853526ccf965a016: done: 0.015093 sec","job_status":"done","queue_duration_s":2.477047,"scheduling_latency_s":2.477067,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015093,"completed_at":"2026-07-09T06:51:26.005Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.620122] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15ac03274ab740748918cd06","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15ac03274ab740748918cd06: start","job_status":"start","queue_duration_s":2.494166,"scheduling_latency_s":2.494204} gitlab # [ 159.621493] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15ac03274ab740748918cd06","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15ac03274ab740748918cd06: done: 0.018292 sec","job_status":"done","queue_duration_s":2.494166,"scheduling_latency_s":2.494204,"redis_calls":13,"redis_duration_s":0.003144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018292,"completed_at":"2026-07-09T06:51:26.025Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.692265] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4e0e2468602b80fb6e1dff1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4e0e2468602b80fb6e1dff1: start","job_status":"start","queue_duration_s":2.513714,"scheduling_latency_s":2.513735} gitlab # [ 159.693966] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4e0e2468602b80fb6e1dff1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4e0e2468602b80fb6e1dff1: done: 0.014778 sec","job_status":"done","queue_duration_s":2.513714,"scheduling_latency_s":2.513735,"redis_calls":13,"redis_duration_s":0.00282,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014778,"completed_at":"2026-07-09T06:51:26.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.778096] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c8c3af900516543bb7d70c6","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c8c3af900516543bb7d70c6: start","job_status":"start","queue_duration_s":2.529759,"scheduling_latency_s":2.529779} gitlab # [ 160.069766] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1537438186 wd_nsec: 1537437958 gitlab # [ 159.988636] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c8c3af900516543bb7d70c6","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c8c3af900516543bb7d70c6: done: 2.748264 sec","job_status":"done","queue_duration_s":2.529759,"scheduling_latency_s":2.529779,"redis_calls":13,"redis_duration_s":2.72445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":2.721855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.748264,"completed_at":"2026-07-09T06:51:28.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.084759] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d85734382cce45a4f50c2c9a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d85734382cce45a4f50c2c9a: start","job_status":"start","queue_duration_s":5.280688,"scheduling_latency_s":5.28073} gitlab # [ 160.103943] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d85734382cce45a4f50c2c9a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d85734382cce45a4f50c2c9a: done: 0.0208 sec","job_status":"done","queue_duration_s":5.280688,"scheduling_latency_s":5.28073,"redis_calls":13,"redis_duration_s":0.0048000000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003274,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0208,"completed_at":"2026-07-09T06:51:28.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.164424] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11d383d8ebb51b1a2953f370","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11d383d8ebb51b1a2953f370: start","job_status":"start","queue_duration_s":5.303163,"scheduling_latency_s":5.303183} gitlab # [ 160.165796] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11d383d8ebb51b1a2953f370","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11d383d8ebb51b1a2953f370: done: 0.01816 sec","job_status":"done","queue_duration_s":5.303163,"scheduling_latency_s":5.303183,"redis_calls":13,"redis_duration_s":0.005927,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002339,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003588,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01816,"completed_at":"2026-07-09T06:51:28.834Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.235537] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f5f5505d7e42100c134bd93","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f5f5505d7e42100c134bd93: start","job_status":"start","queue_duration_s":5.323253,"scheduling_latency_s":5.323274} gitlab # [ 160.236949] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f5f5505d7e42100c134bd93","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f5f5505d7e42100c134bd93: done: 0.018006 sec","job_status":"done","queue_duration_s":5.323253,"scheduling_latency_s":5.323274,"redis_calls":13,"redis_duration_s":0.005594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001688,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018006,"completed_at":"2026-07-09T06:51:28.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.254315] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f4a14946653c0ecc09995fd","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f4a14946653c0ecc09995fd: start","job_status":"start","queue_duration_s":5.343191,"scheduling_latency_s":5.343211} gitlab # [ 160.321754] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f4a14946653c0ecc09995fd","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f4a14946653c0ecc09995fd: done: 0.018022 sec","job_status":"done","queue_duration_s":5.343191,"scheduling_latency_s":5.343211,"redis_calls":13,"redis_duration_s":0.005545,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018022,"completed_at":"2026-07-09T06:51:28.874Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.334158] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"102d700b81c42e0ab86da763","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-102d700b81c42e0ab86da763: start","job_status":"start","queue_duration_s":5.36301,"scheduling_latency_s":5.36303} gitlab # [ 160.418443] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"102d700b81c42e0ab86da763","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-102d700b81c42e0ab86da763: done: 0.018037 sec","job_status":"done","queue_duration_s":5.36301,"scheduling_latency_s":5.36303,"redis_calls":13,"redis_duration_s":0.005637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003801,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018037,"completed_at":"2026-07-09T06:51:28.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.420588] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bad3fca473a5cf0c950871d2","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bad3fca473a5cf0c950871d2: start","job_status":"start","queue_duration_s":5.382963,"scheduling_latency_s":5.382983} gitlab # [ 160.433549] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bad3fca473a5cf0c950871d2","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bad3fca473a5cf0c950871d2: done: 0.017771 sec","job_status":"done","queue_duration_s":5.382963,"scheduling_latency_s":5.382983,"redis_calls":13,"redis_duration_s":0.005244,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003503,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017771,"completed_at":"2026-07-09T06:51:28.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.514473] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bce18816cdf0e0be31290e3","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bce18816cdf0e0be31290e3: start","job_status":"start","queue_duration_s":5.402446,"scheduling_latency_s":5.402513} gitlab # [ 160.516171] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bce18816cdf0e0be31290e3","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bce18816cdf0e0be31290e3: done: 0.018373 sec","job_status":"done","queue_duration_s":5.402446,"scheduling_latency_s":5.402513,"redis_calls":13,"redis_duration_s":0.0057669999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001682,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018373,"completed_at":"2026-07-09T06:51:28.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.597120] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cdae44d0fc6f4f1a1883505","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cdae44d0fc6f4f1a1883505: start","job_status":"start","queue_duration_s":5.422853,"scheduling_latency_s":5.422874} gitlab # [ 160.598321] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cdae44d0fc6f4f1a1883505","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cdae44d0fc6f4f1a1883505: done: 0.027955 sec","job_status":"done","queue_duration_s":5.422853,"scheduling_latency_s":5.422874,"redis_calls":13,"redis_duration_s":0.0056630000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00395,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027955,"completed_at":"2026-07-09T06:51:28.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.667074] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"338889a05e8dc6cdd33f5269","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-338889a05e8dc6cdd33f5269: start","job_status":"start","queue_duration_s":5.453284,"scheduling_latency_s":5.453327} gitlab # [ 160.672311] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"338889a05e8dc6cdd33f5269","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-338889a05e8dc6cdd33f5269: done: 0.020417 sec","job_status":"done","queue_duration_s":5.453284,"scheduling_latency_s":5.453327,"redis_calls":13,"redis_duration_s":0.003018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020417,"completed_at":"2026-07-09T06:51:28.986Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.743136] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.988Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b8de33e496fc322fc7c325d7","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b8de33e496fc322fc7c325d7: start","job_status":"start","queue_duration_s":4.040666,"scheduling_latency_s":4.040686} gitlab # [ 160.810269] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.003Z","message":"No existing merge request to be cleaned up."} gitlab # [ 160.810994] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.017Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b8de33e496fc322fc7c325d7","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b8de33e496fc322fc7c325d7: done: 0.028943 sec","job_status":"done","queue_duration_s":4.040666,"scheduling_latency_s":4.040686,"redis_calls":13,"redis_duration_s":0.003119,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001284,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001374,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000436089,"duration_s":0.028943,"completed_at":"2026-07-09T06:51:29.017Z","load_balancing_strategy":"primary","db_duration_s":0.005962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.814052] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.019Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38ccdc4a060495e49d56db03","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38ccdc4a060495e49d56db03: start","job_status":"start","queue_duration_s":4.071574,"scheduling_latency_s":4.071596} gitlab # [ 160.833207] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.030Z","message":"No existing merge request to be cleaned up."} gitlab # [ 160.836052] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.038Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38ccdc4a060495e49d56db03","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38ccdc4a060495e49d56db03: done: 0.019516 sec","job_status":"done","queue_duration_s":4.071574,"scheduling_latency_s":4.071596,"redis_calls":10,"redis_duration_s":0.004029,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001446,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002583,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019516,"completed_at":"2026-07-09T06:51:29.038Z","load_balancing_strategy":"primary","db_duration_s":0.002667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.919102] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.040Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8a565d7359f533f33d2e14d2","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8a565d7359f533f33d2e14d2: start","job_status":"start","queue_duration_s":4.092539,"scheduling_latency_s":4.092559} gitlab # [ 160.920383] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.049Z","message":"No existing merge request to be cleaned up."} gitlab # [ 160.921299] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.055Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8a565d7359f533f33d2e14d2","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8a565d7359f533f33d2e14d2: done: 0.015923 sec","job_status":"done","queue_duration_s":4.092539,"scheduling_latency_s":4.092559,"redis_calls":10,"redis_duration_s":0.0024720000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001544,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015923,"completed_at":"2026-07-09T06:51:29.055Z","load_balancing_strategy":"primary","db_duration_s":0.001954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.012117] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.057Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cdf6fcd7662562740a4f4044","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cdf6fcd7662562740a4f4044: start","job_status":"start","queue_duration_s":4.109756,"scheduling_latency_s":4.109776} gitlab # [ 161.013107] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.066Z","message":"No existing merge request to be cleaned up."} gitlab # [ 161.013982] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.072Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cdf6fcd7662562740a4f4044","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cdf6fcd7662562740a4f4044: done: 0.015352 sec","job_status":"done","queue_duration_s":4.109756,"scheduling_latency_s":4.109776,"redis_calls":10,"redis_duration_s":0.002524,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00148,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015352,"completed_at":"2026-07-09T06:51:29.072Z","load_balancing_strategy":"primary","db_duration_s":0.001528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.812125] setup-gitlab-start[2748]: W, [2026-07-09T06:51:40.614493 #2748] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 194.047240] runuser[2747]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 194.051842] setup-gitlab-start[2968]: +++ date -d '+7 days' +%F gitlab # [ 194.056101] setup-gitlab-start[2967]: ++ jq -r .token gitlab # [ 194.059537] setup-gitlab-start[2966]: ++ curl -fs -X POST -H 'Authorization: Bearer 1ada74272ecd2e25f1eb27e88b9ead10e8deb7835364224985b565ea2981f204' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-16"}' http://gitlab/api/v4/users/1/personal_access_tokens gitlab # [ 194.526216] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:03.328Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2e28f00dfbd44ec845d9f76f","created_at":"2026-07-09T06:52:03.279Z","trace_propagation_headers":{"sentry-trace":"061fb86fd6484e9ba28c8e8040eccec1-dcd0ae1de21e48f2","baggage":"sentry-trace_id=061fb86fd6484e9ba28c8e8040eccec1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMDPE961CPQ6NT2P4CPQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:f204d91a34b669040ce6ef9ba5775f36f95b247b33712ac224f36df15364d890","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:03.321Z","job_size_bytes":451,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2e28f00dfbd44ec845d9f76f: start","job_status":"start","queue_duration_s":0.006428,"scheduling_latency_s":0.006492} gitlab # [ 194.537856] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:03 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 469 gitlab # [ 194.592582] setup-gitlab-start[2706]: + TOKEN=glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj gitlab # [ 194.594391] setup-gitlab-start[2706]: + test -n glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj gitlab # [ 194.595960] setup-gitlab-start[2706]: + test glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj '!=' null gitlab # [ 194.597228] setup-gitlab-start[2706]: + mkdir -p /var/lib/secrets gitlab # [ 194.599104] setup-gitlab-start[2706]: + echo glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj gitlab # [ 194.600371] setup-gitlab-start[2706]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 194.602131] setup-gitlab-start[2706]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 194.753776] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:03.556Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2e28f00dfbd44ec845d9f76f","created_at":"2026-07-09T06:52:03.279Z","trace_propagation_headers":{"sentry-trace":"061fb86fd6484e9ba28c8e8040eccec1-dcd0ae1de21e48f2","baggage":"sentry-trace_id=061fb86fd6484e9ba28c8e8040eccec1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMDPE961CPQ6NT2P4CPQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:f204d91a34b669040ce6ef9ba5775f36f95b247b33712ac224f36df15364d890","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:03.321Z","job_size_bytes":451,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2e28f00dfbd44ec845d9f76f: fail: 0.227483 sec","job_status":"fail","queue_duration_s":0.006428,"scheduling_latency_s":0.006492,"redis_calls":11,"redis_duration_s":0.0059180000000000005,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001158,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003937,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000823,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00112193,"duration_s":0.227483,"completed_at":"2026-07-09T06:52:03.555Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.042849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.735332] gitaly[1397]: time="2026-07-09T06:52:05.537Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.537" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.740198] gitaly[1397]: time="2026-07-09T06:52:05.538Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.537" grpc.time_ms=0.432 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.924720] gitaly[1397]: time="2026-07-09T06:52:05.728Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.196" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.727" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.929317] gitaly[1397]: time="2026-07-09T06:52:05.728Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.196" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.727" grpc.time_ms=0.61 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.970421] gitaly[1397]: time="2026-07-09T06:52:05.773Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A diskcache=c5eb5297-68f8-4cac-90e4-516adf58383f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-09T06:52:15.210" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.741" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.973638] gitaly[1397]: time="2026-07-09T06:52:05.774Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-09T06:52:15.210" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.741" grpc.time_ms=32.721 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.988611] gitaly[1397]: time="2026-07-09T06:52:05.791Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.260" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.791" pid=1397 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 196.989938] gitaly[1397]: time="2026-07-09T06:52:05.792Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.260" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.791" grpc.time_ms=0.602 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.002164] gitaly[1397]: time="2026-07-09T06:52:05.805Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:15.267" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.797" grpc.time_ms=7.565 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.076366] gitaly[1397]: time="2026-07-09T06:52:05.852Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.852" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.077786] gitaly[1397]: time="2026-07-09T06:52:05.852Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.852" grpc.time_ms=0.29 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.079119] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:05.836Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"31c67b912985432b0968368c","created_at":"2026-07-09T06:52:05.818Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:05.834Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-31c67b912985432b0968368c: start","job_status":"start","queue_duration_s":0.002522,"scheduling_latency_s":0.002578} gitlab # [ 197.117618] gitaly[1397]: time="2026-07-09T06:52:05.921Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A diskcache=383d173d-a884-474e-ab0c-e0833a9751b0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-09T06:52:15.005" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.855" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.118957] gitaly[1397]: time="2026-07-09T06:52:05.921Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=32 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-09T06:52:15.005" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.855" grpc.time_ms=65.859 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.126472] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:05.929Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"31c67b912985432b0968368c","created_at":"2026-07-09T06:52:05.818Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:05.834Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-31c67b912985432b0968368c: done: 0.09332 sec","job_status":"done","queue_duration_s":0.002522,"scheduling_latency_s":0.002578,"redis_calls":10,"redis_duration_s":0.003646,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001829,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001817,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.068455907,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.09332,"completed_at":"2026-07-09T06:52:05.929Z","load_balancing_strategy":"primary","db_duration_s":0.010906,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 197.217091] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:05.931Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7b19655c50d8c035edb7a2ce","created_at":"2026-07-09T06:52:05.840Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:05.848Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7b19655c50d8c035edb7a2ce: start","job_status":"start","queue_duration_s":0.083179,"scheduling_latency_s":0.083201} gitlab # [ 197.218250] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.009Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7b19655c50d8c035edb7a2ce","created_at":"2026-07-09T06:52:05.840Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:05.848Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7b19655c50d8c035edb7a2ce: done: 0.078545 sec","job_status":"done","queue_duration_s":0.083179,"scheduling_latency_s":0.083201,"redis_calls":14,"redis_duration_s":0.004509,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000972,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002453,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.055458166,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.078545,"completed_at":"2026-07-09T06:52:06.009Z","load_balancing_strategy":"primary","db_duration_s":0.017648,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 197.293412] gitaly[1397]: time="2026-07-09T06:52:05.932Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.081" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.932" pid=1397 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 197.294782] gitaly[1397]: time="2026-07-09T06:52:05.932Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.081" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.932" grpc.time_ms=0.433 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.303722] gitaly[1397]: time="2026-07-09T06:52:05.955Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:15.089" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.940" grpc.time_ms=14.619 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.325906] gitaly[1397]: time="2026-07-09T06:52:05.977Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.126" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.977" grpc.time_ms=0.21 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.404080] gitaly[1397]: time="2026-07-09T06:52:05.984Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-09T06:52:15.133" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.984" grpc.time_ms=0.23 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.405409] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.124Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"98847b990321840b0384c3f1","created_at":"2026-07-09T06:52:06.117Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.123Z","job_size_bytes":5,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-98847b990321840b0384c3f1: start","job_status":"start","queue_duration_s":0.001533,"scheduling_latency_s":0.00157} gitlab # [ 197.406200] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.198Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"98847b990321840b0384c3f1","created_at":"2026-07-09T06:52:06.117Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.123Z","job_size_bytes":5,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-98847b990321840b0384c3f1: done: 0.073765 sec","job_status":"done","queue_duration_s":0.001533,"scheduling_latency_s":0.00157,"redis_calls":12,"redis_duration_s":0.0029980000000000002,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000759,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001775,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000393905,"duration_s":0.073765,"completed_at":"2026-07-09T06:52:06.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010177,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 197.477261] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.200Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"cca36d43cf838746f70b6a58","created_at":"2026-07-09T06:52:06.143Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.147Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-cca36d43cf838746f70b6a58: start","job_status":"start","queue_duration_s":0.052521,"scheduling_latency_s":0.052542} gitlab # [ 197.478801] gitaly[1397]: time="2026-07-09T06:52:06.171Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:16.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:06.164" grpc.time_ms=6.171 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.602564] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.405Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"cca36d43cf838746f70b6a58","created_at":"2026-07-09T06:52:06.143Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.147Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-cca36d43cf838746f70b6a58: done: 0.20575 sec","job_status":"done","queue_duration_s":0.052521,"scheduling_latency_s":0.052542,"redis_calls":11,"redis_duration_s":0.002745,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000699,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001558,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.178,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.178,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.160493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346971,"duration_s":0.20575,"completed_at":"2026-07-09T06:52:06.405Z","load_balancing_strategy":"primary","db_duration_s":0.043868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.604339] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.407Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"1e89820572d8eaf163cf445e","created_at":"2026-07-09T06:52:06.149Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.153Z","job_size_bytes":89,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-1e89820572d8eaf163cf445e: start","job_status":"start","queue_duration_s":0.253797,"scheduling_latency_s":0.253818} gitlab # [ 197.686819] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.454Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"1e89820572d8eaf163cf445e","created_at":"2026-07-09T06:52:06.149Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.153Z","job_size_bytes":89,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-1e89820572d8eaf163cf445e: done: 0.047462 sec","job_status":"done","queue_duration_s":0.253797,"scheduling_latency_s":0.253818,"redis_calls":11,"redis_duration_s":0.002551,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00066,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000447,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000353118,"duration_s":0.047462,"completed_at":"2026-07-09T06:52:06.454Z","load_balancing_strategy":"primary","db_duration_s":0.014696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.707646] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:06 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 3101 gitlab # [ 197.709510] setup-gitlab-start[2706]: + mkdir -p /tmp/test-flake gitlab # [ 197.710852] setup-gitlab-start[2706]: + cd /tmp/test-flake gitlab # [ 197.711971] setup-gitlab-start[2706]: + git init -b master gitlab # [ 197.772233] setup-gitlab-start[3012]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 197.773458] setup-gitlab-start[2706]: + git config user.name test gitlab # [ 197.777883] setup-gitlab-start[2706]: + git config user.email test@example.com gitlab # [ 197.781284] setup-gitlab-start[2706]: + cat gitlab # [ 197.783347] setup-gitlab-start[2706]: + git add flake.nix gitlab # [ 197.787326] setup-gitlab-start[2706]: + git commit -m 'initial commit' gitlab # [ 197.794353] setup-gitlab-start[3017]: [master (root-commit) 8cb3788] initial commit gitlab # [ 197.796126] setup-gitlab-start[3017]: 1 file changed, 10 insertions(+) gitlab # [ 197.797976] setup-gitlab-start[3017]: create mode 100644 flake.nix gitlab # [ 197.800236] setup-gitlab-start[2706]: + git remote add origin http://oauth2:glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj@gitlab/root/test-flake.git gitlab # [ 197.804272] setup-gitlab-start[2706]: + git push -u origin master gitlab # [ 197.865217] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"24e96ccdafcd80ec3b3d3341","created_at":"2026-07-09T06:52:06.661Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad9d3e47d7624121189f743f3a202b59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-09T06:52:06.665Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-24e96ccdafcd80ec3b3d3341: start","job_status":"start","queue_duration_s":0.001823,"scheduling_latency_s":0.001919} gitlab # [ 197.923965] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"24e96ccdafcd80ec3b3d3341","created_at":"2026-07-09T06:52:06.661Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad9d3e47d7624121189f743f3a202b59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-09T06:52:06.665Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-24e96ccdafcd80ec3b3d3341: done: 0.059511 sec","job_status":"done","queue_duration_s":0.001823,"scheduling_latency_s":0.001919,"redis_calls":9,"redis_duration_s":0.009823,"redis_read_bytes":7,"redis_write_bytes":1018,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00103,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":277,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001664,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059511,"completed_at":"2026-07-09T06:52:06.727Z","load_balancing_strategy":"primary","db_duration_s":0.005337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.943615] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:06 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 103 gitlab # [ 197.954100] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.756Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9147061b0f9adf512b97c1b7","created_at":"2026-07-09T06:52:06.696Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4504040db5ff04f5464f6426c388c90","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.703Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9147061b0f9adf512b97c1b7: start","job_status":"start","queue_duration_s":0.053241,"scheduling_latency_s":0.053275} gitlab # [ 197.974774] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.777Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 198.013697] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9147061b0f9adf512b97c1b7","created_at":"2026-07-09T06:52:06.696Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4504040db5ff04f5464f6426c388c90","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.703Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9147061b0f9adf512b97c1b7: done: 0.058314 sec","job_status":"done","queue_duration_s":0.053241,"scheduling_latency_s":0.053275,"redis_calls":7,"redis_duration_s":0.014950000000000001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014467,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058314,"completed_at":"2026-07-09T06:52:06.814Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.021128] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.821Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a30e018c14ef1fd4b4235201","created_at":"2026-07-09T06:52:06.718Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c36be163c6308060e191ff086c3561d6","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.722Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a30e018c14ef1fd4b4235201: start","job_status":"start","queue_duration_s":0.098891,"scheduling_latency_s":0.098914} gitlab # [ 198.060138] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a30e018c14ef1fd4b4235201","created_at":"2026-07-09T06:52:06.718Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c36be163c6308060e191ff086c3561d6","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.722Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a30e018c14ef1fd4b4235201: done: 0.041839 sec","job_status":"done","queue_duration_s":0.098891,"scheduling_latency_s":0.098914,"redis_calls":10,"redis_duration_s":0.007405,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002224,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005181,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.015120916,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.041839,"completed_at":"2026-07-09T06:52:06.863Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014564,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 198.158781] gitaly[1397]: time="2026-07-09T06:52:06.903Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBR6W49PB5QEQQHYW8E87 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:06.887" grpc.time_ms=15.636 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.161415] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:06 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 156 gitlab # [ 198.163469] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.864Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"75a448ce9f4450ea35a658f3","created_at":"2026-07-09T06:52:06.753Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6b156631dc9e6c7de6cf829c670a11b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.760Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-75a448ce9f4450ea35a658f3: start","job_status":"start","queue_duration_s":0.103998,"scheduling_latency_s":0.104018} gitlab # [ 198.166157] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.883Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"75a448ce9f4450ea35a658f3","created_at":"2026-07-09T06:52:06.753Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6b156631dc9e6c7de6cf829c670a11b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.760Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-75a448ce9f4450ea35a658f3: done: 0.018766 sec","job_status":"done","queue_duration_s":0.103998,"scheduling_latency_s":0.104018,"redis_calls":12,"redis_duration_s":0.003625,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000876,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000636,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000387759,"duration_s":0.018766,"completed_at":"2026-07-09T06:52:06.883Z","load_balancing_strategy":"primary","db_duration_s":0.000541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.235146] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"9d414adfc92895953dd42831","created_at":"2026-07-09T06:52:06.771Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cedb24bb71d4de55bd0f8403bd86204","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.783Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-9d414adfc92895953dd42831: start","job_status":"start","queue_duration_s":0.10078,"scheduling_latency_s":0.100801} gitlab # [ 198.236397] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"9d414adfc92895953dd42831","created_at":"2026-07-09T06:52:06.771Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cedb24bb71d4de55bd0f8403bd86204","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.783Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-9d414adfc92895953dd42831: done: 0.018077 sec","job_status":"done","queue_duration_s":0.10078,"scheduling_latency_s":0.100801,"redis_calls":7,"redis_duration_s":0.00481,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018077,"completed_at":"2026-07-09T06:52:06.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.312315] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a96ba83ac23398c35597ba0f","created_at":"2026-07-09T06:52:06.798Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca07d2cc46704fac8b9096172ea0bd07","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.801Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a96ba83ac23398c35597ba0f: start","job_status":"start","queue_duration_s":0.103058,"scheduling_latency_s":0.103078} gitlab # [ 198.322645] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.952Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a96ba83ac23398c35597ba0f","created_at":"2026-07-09T06:52:06.798Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca07d2cc46704fac8b9096172ea0bd07","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.801Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a96ba83ac23398c35597ba0f: done: 0.048228 sec","job_status":"done","queue_duration_s":0.103058,"scheduling_latency_s":0.103078,"redis_calls":9,"redis_duration_s":0.003617,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002542,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000498388,"exclusive_lock_hold_duration_s":0.004753397,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.048228,"completed_at":"2026-07-09T06:52:06.952Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003514,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.435466] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cd4d1da12d4e6195d80ade08","created_at":"2026-07-09T06:52:06.812Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"336ee9b3f461fe1b45a493565957a5e3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.816Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cd4d1da12d4e6195d80ade08: start","job_status":"start","queue_duration_s":0.145634,"scheduling_latency_s":0.145656} gitlab # [ 198.436781] gitaly[1397]: time="2026-07-09T06:52:07.123Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:17.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:07.112" grpc.time_ms=11.023 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.440822] gitaly[1397]: time="2026-07-09T06:52:07.162Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-09T06:52:37.053" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-09T06:52:07.153" grpc.time_ms=8.263 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.449893] gitaly[1397]: time="2026-07-09T06:52:07.180Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV duration_ms=121ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 198.460104] gitaly[1397]: time="2026-07-09T06:52:07.262Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV duration_ms=81ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 198.460852] gitaly[1397]: time="2026-07-09T06:52:07.263Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.058" grpc.time_ms=204.574 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.541926] gitaly[1397]: time="2026-07-09T06:52:07.292Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.292" grpc.time_ms=0.606 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.546193] gitaly[1397]: time="2026-07-09T06:52:07.323Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.323" grpc.time_ms=0.455 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.553680] gitaly[1397]: time="2026-07-09T06:52:07.356Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.352" grpc.time_ms=2.755 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.650157] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cd4d1da12d4e6195d80ade08","created_at":"2026-07-09T06:52:06.812Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"336ee9b3f461fe1b45a493565957a5e3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.816Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cd4d1da12d4e6195d80ade08: done: 0.491018 sec","job_status":"done","queue_duration_s":0.145634,"scheduling_latency_s":0.145656,"redis_calls":571,"redis_duration_s":0.209126,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.207898,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.305676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.491018,"completed_at":"2026-07-09T06:52:07.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.657322] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"bd86919a48e36e2aa32394a6","created_at":"2026-07-09T06:52:06.828Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"006beaa50655fce307bc68cb594185ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.833Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-bd86919a48e36e2aa32394a6: start","job_status":"start","queue_duration_s":0.621287,"scheduling_latency_s":0.621315} gitlab # [ 198.673167] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"bd86919a48e36e2aa32394a6","created_at":"2026-07-09T06:52:06.828Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"006beaa50655fce307bc68cb594185ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.833Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-bd86919a48e36e2aa32394a6: done: 0.021822 sec","job_status":"done","queue_duration_s":0.621287,"scheduling_latency_s":0.621315,"redis_calls":9,"redis_duration_s":0.0051660000000000005,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001637,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003529,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000487771,"exclusive_lock_hold_duration_s":0.005938464,"duration_s":0.021822,"completed_at":"2026-07-09T06:52:07.476Z","load_balancing_strategy":"primary","db_duration_s":0.004193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.775885] gitaly[1397]: time="2026-07-09T06:52:07.498Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:17.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:07.477" grpc.time_ms=20.521 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.777359] setup-gitlab-start[3021]: To http://gitlab/root/test-flake.git gitlab # [ 198.778176] setup-gitlab-start[3021]: * [new branch] master -> master gitlab # [ 198.778988] setup-gitlab-start[3021]: branch 'master' set up to track 'origin/master'. gitlab # [ 198.779865] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"48a35d3f5ae2742e1fb675c1","created_at":"2026-07-09T06:52:06.841Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41b76d1e42a2ff3729965f8b3a10acfe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-09T06:52:06.845Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-48a35d3f5ae2742e1fb675c1: start","job_status":"start","queue_duration_s":0.63573,"scheduling_latency_s":0.63576} gitlab # [ 198.780979] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.489Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 198.787132] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.496Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"48a35d3f5ae2742e1fb675c1","created_at":"2026-07-09T06:52:06.841Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41b76d1e42a2ff3729965f8b3a10acfe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-09T06:52:06.845Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-48a35d3f5ae2742e1fb675c1: done: 0.014889 sec","job_status":"done","queue_duration_s":0.63573,"scheduling_latency_s":0.63576,"redis_calls":7,"redis_duration_s":0.003571,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000867,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014889,"completed_at":"2026-07-09T06:52:07.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.871145] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.498Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c60e2a1fad745992f745fa2a","created_at":"2026-07-09T06:52:06.851Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"cb68b841149f620c8a5520fe086b1dad","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.900Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c60e2a1fad745992f745fa2a: start","job_status":"start","queue_duration_s":0.59868,"scheduling_latency_s":0.598701} gitlab # [ 198.872389] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c60e2a1fad745992f745fa2a","created_at":"2026-07-09T06:52:06.851Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"cb68b841149f620c8a5520fe086b1dad","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.900Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c60e2a1fad745992f745fa2a: done: 0.020657 sec","job_status":"done","queue_duration_s":0.59868,"scheduling_latency_s":0.598701,"redis_calls":10,"redis_duration_s":0.002518,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001545,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000393067,"duration_s":0.020657,"completed_at":"2026-07-09T06:52:07.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.938896] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2c5e8926c0dfb0e0842572cd","created_at":"2026-07-09T06:52:06.915Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.918Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2c5e8926c0dfb0e0842572cd: start","job_status":"start","queue_duration_s":0.60455,"scheduling_latency_s":0.604579} gitlab # [ 199.015463] gitlab-workhorse[1306]: time="2026-07-09T06:52:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 199.018578] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:07 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 660 gitlab: (finished: waiting for unit setup-gitlab.service, in 199.95 seconds) gitlab # [ 199.020138] gitaly[1397]: time="2026-07-09T06:52:07.541Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-09T06:52:17.049" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:07.519" grpc.time_ms=21.266 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: waiting for unit nixbot.service gitlab # [ 199.020934] gitaly[1397]: time="2026-07-09T06:52:07.560Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV duration_ms=173ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 199.022460] systemd[1]: Finished setup-gitlab.service. gitlab # [ 199.023497] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2c5e8926c0dfb0e0842572cd","created_at":"2026-07-09T06:52:06.915Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.918Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2c5e8926c0dfb0e0842572cd: done: 0.081951 sec","job_status":"done","queue_duration_s":0.60455,"scheduling_latency_s":0.604579,"redis_calls":60,"redis_duration_s":0.016019000000000002,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000632,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.014394,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.081951,"completed_at":"2026-07-09T06:52:07.605Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.047854] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6f0a81b31ea5c2a95b06971f","created_at":"2026-07-09T06:52:06.922Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8b2b54ab96107ef21b0a651cb04e16b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.932Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6f0a81b31ea5c2a95b06971f: start","job_status":"start","queue_duration_s":0.674654,"scheduling_latency_s":0.674681} gitlab # [ 199.122413] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6f0a81b31ea5c2a95b06971f","created_at":"2026-07-09T06:52:06.922Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8b2b54ab96107ef21b0a651cb04e16b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.932Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6f0a81b31ea5c2a95b06971f: done: 0.190277 sec","job_status":"done","queue_duration_s":0.674654,"scheduling_latency_s":0.674681,"redis_calls":11,"redis_duration_s":0.002704,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000471009,"exclusive_lock_hold_duration_s":0.172752403,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.190277,"completed_at":"2026-07-09T06:52:07.797Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069754,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.133405] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d5ba35489ab709a532a46788","created_at":"2026-07-09T06:52:06.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e03ed705f4c4227e34e9e3996fd20c59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.941Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d5ba35489ab709a532a46788: start","job_status":"start","queue_duration_s":0.857317,"scheduling_latency_s":0.857338} gitlab # [ 199.200094] gitaly[1397]: time="2026-07-09T06:52:07.561Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.386" grpc.time_ms=174.807 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.201325] gitaly[1397]: time="2026-07-09T06:52:07.572Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV diskcache=7c68ebf2-4f0d-4567-8733-75c11e4006fb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:06.994" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.202223] systemd[1]: Starting nixbot CI service... gitlab # [ 199.203419] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.829Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d5ba35489ab709a532a46788","created_at":"2026-07-09T06:52:06.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e03ed705f4c4227e34e9e3996fd20c59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.941Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d5ba35489ab709a532a46788: done: 0.030526 sec","job_status":"done","queue_duration_s":0.857317,"scheduling_latency_s":0.857338,"redis_calls":9,"redis_duration_s":0.002431,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000445866,"exclusive_lock_hold_duration_s":0.016370517,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.030526,"completed_at":"2026-07-09T06:52:07.829Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.290173] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7bae42e74bf38cfe7a0b521d","created_at":"2026-07-09T06:52:06.953Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c07c5301136c2f7fbf48eae3e5c5d5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.955Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7bae42e74bf38cfe7a0b521d: start","job_status":"start","queue_duration_s":0.874865,"scheduling_latency_s":0.874929} gitlab # [ 199.291972] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.839Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 199.293050] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.843Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7bae42e74bf38cfe7a0b521d","created_at":"2026-07-09T06:52:06.953Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c07c5301136c2f7fbf48eae3e5c5d5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.955Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7bae42e74bf38cfe7a0b521d: done: 0.012823 sec","job_status":"done","queue_duration_s":0.874865,"scheduling_latency_s":0.874929,"redis_calls":7,"redis_duration_s":0.0017140000000000002,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000519,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012823,"completed_at":"2026-07-09T06:52:07.843Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.365292] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b1b08eeb3d1a3b70da81fc29","created_at":"2026-07-09T06:52:06.964Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34a5125bdfe2d0db9f53574bd23894d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.965Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b1b08eeb3d1a3b70da81fc29: start","job_status":"start","queue_duration_s":0.878975,"scheduling_latency_s":0.878995} gitlab # [ 199.366854] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b1b08eeb3d1a3b70da81fc29","created_at":"2026-07-09T06:52:06.964Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34a5125bdfe2d0db9f53574bd23894d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.965Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b1b08eeb3d1a3b70da81fc29: done: 0.012189 sec","job_status":"done","queue_duration_s":0.878975,"scheduling_latency_s":0.878995,"redis_calls":7,"redis_duration_s":0.001605,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012189,"completed_at":"2026-07-09T06:52:07.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.435188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b8c4c54c5143f868a236ea2c","created_at":"2026-07-09T06:52:06.989Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d1cdff36c8527dc244a1fc292053a21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.993Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b8c4c54c5143f868a236ea2c: start","job_status":"start","queue_duration_s":0.864423,"scheduling_latency_s":0.864444} gitlab # [ 199.510410] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.865Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 199.511658] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.870Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b8c4c54c5143f868a236ea2c","created_at":"2026-07-09T06:52:06.989Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d1cdff36c8527dc244a1fc292053a21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.993Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b8c4c54c5143f868a236ea2c: done: 0.012069 sec","job_status":"done","queue_duration_s":0.864423,"scheduling_latency_s":0.864444,"redis_calls":7,"redis_duration_s":0.00183,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000671,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012069,"completed_at":"2026-07-09T06:52:07.870Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.517063] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.871Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b6f1a9505451f5807fe61400","created_at":"2026-07-09T06:52:07.002Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3959cca50529f64ccc0b084acb05c6de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-09T06:52:07.006Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b6f1a9505451f5807fe61400: start","job_status":"start","queue_duration_s":0.864962,"scheduling_latency_s":0.864982} gitlab # [ 199.586418] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.879Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 199.587884] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b6f1a9505451f5807fe61400","created_at":"2026-07-09T06:52:07.002Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3959cca50529f64ccc0b084acb05c6de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-09T06:52:07.006Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b6f1a9505451f5807fe61400: done: 0.011651 sec","job_status":"done","queue_duration_s":0.864962,"scheduling_latency_s":0.864982,"redis_calls":7,"redis_duration_s":0.00157,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011651,"completed_at":"2026-07-09T06:52:07.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.661298] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"37c3aa7d2250ada6677f2864","created_at":"2026-07-09T06:52:07.024Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b50b12011fae63fd3c85bfc874b7163c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.035Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-37c3aa7d2250ada6677f2864: start","job_status":"start","queue_duration_s":0.84885,"scheduling_latency_s":0.84887} gitlab # [ 199.662807] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"37c3aa7d2250ada6677f2864","created_at":"2026-07-09T06:52:07.024Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b50b12011fae63fd3c85bfc874b7163c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.035Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-37c3aa7d2250ada6677f2864: done: 0.020691 sec","job_status":"done","queue_duration_s":0.84885,"scheduling_latency_s":0.84887,"redis_calls":12,"redis_duration_s":0.002623,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000443352,"exclusive_lock_hold_duration_s":0.00421953,"duration_s":0.020691,"completed_at":"2026-07-09T06:52:07.905Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.734301] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.906Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c969a5d797899d7398f39983","created_at":"2026-07-09T06:52:07.049Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9967f72573b0284850b7450e4ea63d1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.051Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c969a5d797899d7398f39983: start","job_status":"start","queue_duration_s":0.854784,"scheduling_latency_s":0.854845} gitlab # [ 199.735796] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.923Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c969a5d797899d7398f39983","created_at":"2026-07-09T06:52:07.049Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9967f72573b0284850b7450e4ea63d1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.051Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c969a5d797899d7398f39983: done: 0.016936 sec","job_status":"done","queue_duration_s":0.854784,"scheduling_latency_s":0.854845,"redis_calls":7,"redis_duration_s":0.001701,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016936,"completed_at":"2026-07-09T06:52:07.923Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.803629] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a45375ff7c115532ebcc03a2","created_at":"2026-07-09T06:52:07.079Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.081Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a45375ff7c115532ebcc03a2: start","job_status":"start","queue_duration_s":0.843632,"scheduling_latency_s":0.843653} gitlab # [ 199.806578] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.944Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a45375ff7c115532ebcc03a2","created_at":"2026-07-09T06:52:07.079Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.081Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a45375ff7c115532ebcc03a2: done: 0.019808 sec","job_status":"done","queue_duration_s":0.843632,"scheduling_latency_s":0.843653,"redis_calls":14,"redis_duration_s":0.0031680000000000002,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000348,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002181,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019808,"completed_at":"2026-07-09T06:52:07.944Z","load_balancing_strategy":"primary","db_duration_s":0.000786,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.888300] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.946Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0d4db9210771832a26d4bfd3","created_at":"2026-07-09T06:52:07.085Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf3cffa8950e35caba84b17f727d5b8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.089Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0d4db9210771832a26d4bfd3: start","job_status":"start","queue_duration_s":0.856584,"scheduling_latency_s":0.856605} gitlab # [ 199.889749] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.958Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0d4db9210771832a26d4bfd3","created_at":"2026-07-09T06:52:07.085Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf3cffa8950e35caba84b17f727d5b8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.089Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0d4db9210771832a26d4bfd3: done: 0.012903 sec","job_status":"done","queue_duration_s":0.856584,"scheduling_latency_s":0.856605,"redis_calls":7,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012903,"completed_at":"2026-07-09T06:52:07.958Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000774,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.974301] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.960Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ecbf54ad4f592214ae279d8f","created_at":"2026-07-09T06:52:07.103Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcab947599967a5aa99a0b65ea85ab1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.106Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ecbf54ad4f592214ae279d8f: start","job_status":"start","queue_duration_s":0.85362,"scheduling_latency_s":0.853641} gitlab # [ 199.975837] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.967Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 199.976914] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ecbf54ad4f592214ae279d8f","created_at":"2026-07-09T06:52:07.103Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcab947599967a5aa99a0b65ea85ab1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.106Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ecbf54ad4f592214ae279d8f: done: 0.011425 sec","job_status":"done","queue_duration_s":0.85362,"scheduling_latency_s":0.853641,"redis_calls":7,"redis_duration_s":0.001548,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011425,"completed_at":"2026-07-09T06:52:07.971Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.044307] gitaly[1397]: time="2026-07-09T06:52:07.573Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=141 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13100 command.oublock=32 command.real_time_ms=568 command.spawn_token_fork_ms=1 command.system_time_ms=87 command.user_time_ms=54 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:06.994" grpc.time_ms=578.412 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 200.049424] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a07a7e493076332dca553af8","created_at":"2026-07-09T06:52:07.123Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2a8a018981418ee8313d0f375b84b55","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.131Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a07a7e493076332dca553af8: start","job_status":"start","queue_duration_s":0.84141,"scheduling_latency_s":0.841431} gitlab # [ 200.137953] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a07a7e493076332dca553af8","created_at":"2026-07-09T06:52:07.123Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2a8a018981418ee8313d0f375b84b55","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.131Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a07a7e493076332dca553af8: done: 0.024419 sec","job_status":"done","queue_duration_s":0.84141,"scheduling_latency_s":0.841431,"redis_calls":9,"redis_duration_s":0.002016,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000516546,"exclusive_lock_hold_duration_s":0.007834515,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.024419,"completed_at":"2026-07-09T06:52:07.997Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.142194] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0fae6701baf2f53220866eb7","created_at":"2026-07-09T06:52:07.164Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e47fe783120b551b611246f21acb64d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.167Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0fae6701baf2f53220866eb7: start","job_status":"start","queue_duration_s":0.830754,"scheduling_latency_s":0.830774} gitlab # [ 200.213221] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.014Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0fae6701baf2f53220866eb7","created_at":"2026-07-09T06:52:07.164Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e47fe783120b551b611246f21acb64d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.167Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0fae6701baf2f53220866eb7: done: 0.016134 sec","job_status":"done","queue_duration_s":0.830754,"scheduling_latency_s":0.830774,"redis_calls":7,"redis_duration_s":0.001771,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.016134,"completed_at":"2026-07-09T06:52:08.014Z","load_balancing_strategy":"primary","db_duration_s":0.00404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.285836] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.016Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"7136ba77f6fc3dccf678b89d","created_at":"2026-07-09T06:52:07.174Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ca8838d7194ae78aaa4879b40252bb3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.181Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-7136ba77f6fc3dccf678b89d: start","job_status":"start","queue_duration_s":0.834903,"scheduling_latency_s":0.834923} gitlab # [ 200.287108] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"7136ba77f6fc3dccf678b89d","created_at":"2026-07-09T06:52:07.174Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ca8838d7194ae78aaa4879b40252bb3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.181Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-7136ba77f6fc3dccf678b89d: done: 0.020348 sec","job_status":"done","queue_duration_s":0.834903,"scheduling_latency_s":0.834923,"redis_calls":10,"redis_duration_s":0.002136,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337194,"duration_s":0.020348,"completed_at":"2026-07-09T06:52:08.036Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.309496] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.037Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 8cb378872b8fe1f7cde149c561f5802ee16ad182 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e86a6a54322f311f46ee7687","created_at":"2026-07-09T06:52:07.421Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-f59c66934d58426d","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.443Z","job_size_bytes":185,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e86a6a54322f311f46ee7687: start","job_status":"start","queue_duration_s":0.59383,"scheduling_latency_s":0.59385} gitlab # [ 200.382511] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.056Z","message":"changes: \"0000000000000000000000000000000000000000 8cb378872b8fe1f7cde149c561f5802ee16ad182 refs/heads/master\\n\""} gitlab # [ 200.383994] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.721Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 8cb378872b8fe1f7cde149c561f5802ee16ad182 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e86a6a54322f311f46ee7687","created_at":"2026-07-09T06:52:07.421Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-f59c66934d58426d","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.443Z","job_size_bytes":185,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e86a6a54322f311f46ee7687: done: 0.683293 sec","job_status":"done","queue_duration_s":0.59383,"scheduling_latency_s":0.59385,"gitaly_calls":13,"gitaly_duration_s":0.161524,"redis_calls":201,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.043203000000000005,"redis_read_bytes":963,"redis_write_bytes":20009,"redis_cache_calls":14,"redis_cache_duration_s":0.00357,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":34,"redis_feature_flag_duration_s":0.007701,"redis_feature_flag_read_bytes":542,"redis_feature_flag_write_bytes":4562,"redis_queues_calls":8,"redis_queues_duration_s":0.001118,"redis_queues_read_bytes":12,"redis_queues_write_bytes":4932,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00201,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.018138,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.010666,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":53,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":53,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":53,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.25,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.267,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.432393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000498108,"exclusive_lock_hold_duration_s":0.0330285,"duration_s":0.683293,"completed_at":"2026-07-09T06:52:08.721Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.068602,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 200.463108] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcf8e37c735507af83c31d11","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcf8e37c735507af83c31d11: start","job_status":"start","queue_duration_s":1.123703,"scheduling_latency_s":1.123726} gitlab # [ 200.464295] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcf8e37c735507af83c31d11","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcf8e37c735507af83c31d11: done: 0.029111 sec","job_status":"done","queue_duration_s":1.123703,"scheduling_latency_s":1.123726,"redis_calls":26,"redis_duration_s":0.005818,"redis_read_bytes":150,"redis_write_bytes":2841,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.002882,"redis_feature_flag_read_bytes":140,"redis_feature_flag_write_bytes":1684,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029111,"completed_at":"2026-07-09T06:52:08.751Z","load_balancing_strategy":"primary","db_duration_s":0.001915,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.499423] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38b0612f1b1c3eae6914c356","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38b0612f1b1c3eae6914c356: start","job_status":"start","queue_duration_s":1.154116,"scheduling_latency_s":1.154137} gitlab # [ 200.582376] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38b0612f1b1c3eae6914c356","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38b0612f1b1c3eae6914c356: done: 0.015283 sec","job_status":"done","queue_duration_s":1.154116,"scheduling_latency_s":1.154137,"redis_calls":13,"redis_duration_s":0.002778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001872,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015283,"completed_at":"2026-07-09T06:52:08.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.584164] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0220c5ff88f5ba5aa3d459dd","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0220c5ff88f5ba5aa3d459dd: start","job_status":"start","queue_duration_s":1.170887,"scheduling_latency_s":1.170907} gitlab # [ 200.598138] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0220c5ff88f5ba5aa3d459dd","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0220c5ff88f5ba5aa3d459dd: done: 0.015573 sec","job_status":"done","queue_duration_s":1.170887,"scheduling_latency_s":1.170907,"redis_calls":13,"redis_duration_s":0.002873,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015573,"completed_at":"2026-07-09T06:52:08.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.677131] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7541d34c337bb99737bba584","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7541d34c337bb99737bba584: start","job_status":"start","queue_duration_s":1.18793,"scheduling_latency_s":1.187951} gitlab # [ 200.678438] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7541d34c337bb99737bba584","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7541d34c337bb99737bba584: done: 0.01556 sec","job_status":"done","queue_duration_s":1.18793,"scheduling_latency_s":1.187951,"redis_calls":13,"redis_duration_s":0.002731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01556,"completed_at":"2026-07-09T06:52:08.802Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.747214] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54a6f79d20287855d0a4edf0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54a6f79d20287855d0a4edf0: start","job_status":"start","queue_duration_s":1.205294,"scheduling_latency_s":1.205314} gitlab # [ 200.756045] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54a6f79d20287855d0a4edf0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54a6f79d20287855d0a4edf0: done: 0.015179 sec","job_status":"done","queue_duration_s":1.205294,"scheduling_latency_s":1.205314,"redis_calls":13,"redis_duration_s":0.002817,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015179,"completed_at":"2026-07-09T06:52:08.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.826302] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"728f56c6aea63a6e4d29afb7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-728f56c6aea63a6e4d29afb7: start","job_status":"start","queue_duration_s":1.221708,"scheduling_latency_s":1.221729} gitlab # [ 200.834796] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"728f56c6aea63a6e4d29afb7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-728f56c6aea63a6e4d29afb7: done: 0.015049 sec","job_status":"done","queue_duration_s":1.221708,"scheduling_latency_s":1.221729,"redis_calls":13,"redis_duration_s":0.003183,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002314,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015049,"completed_at":"2026-07-09T06:52:08.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.915775] gitaly[1397]: time="2026-07-09T06:52:08.097Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:18.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.086" grpc.time_ms=10.271 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 200.916906] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c217cd1faa8a6db01a5961eb","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c217cd1faa8a6db01a5961eb: start","job_status":"start","queue_duration_s":1.238017,"scheduling_latency_s":1.238037} gitlab # [ 200.994942] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c217cd1faa8a6db01a5961eb","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c217cd1faa8a6db01a5961eb: done: 0.015182 sec","job_status":"done","queue_duration_s":1.238017,"scheduling_latency_s":1.238037,"redis_calls":13,"redis_duration_s":0.002886,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015182,"completed_at":"2026-07-09T06:52:08.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.996308] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89a22bcf7bc1d2e7aa60cf51","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89a22bcf7bc1d2e7aa60cf51: start","job_status":"start","queue_duration_s":1.25446,"scheduling_latency_s":1.254491} gitlab # [ 201.062824] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89a22bcf7bc1d2e7aa60cf51","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89a22bcf7bc1d2e7aa60cf51: done: 0.015291 sec","job_status":"done","queue_duration_s":1.25446,"scheduling_latency_s":1.254491,"redis_calls":13,"redis_duration_s":0.002862,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015291,"completed_at":"2026-07-09T06:52:08.868Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.130435] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd30fc4d1ef5923e07020ccc","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd30fc4d1ef5923e07020ccc: start","job_status":"start","queue_duration_s":1.271454,"scheduling_latency_s":1.271475} gitlab # [ 201.131692] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd30fc4d1ef5923e07020ccc","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd30fc4d1ef5923e07020ccc: done: 0.016542 sec","job_status":"done","queue_duration_s":1.271454,"scheduling_latency_s":1.271475,"redis_calls":13,"redis_duration_s":0.003261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016542,"completed_at":"2026-07-09T06:52:08.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.199510] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d72352fa1815e50fc32ee36","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d72352fa1815e50fc32ee36: start","job_status":"start","queue_duration_s":1.289282,"scheduling_latency_s":1.289303} gitlab # [ 201.201049] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d72352fa1815e50fc32ee36","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d72352fa1815e50fc32ee36: done: 0.015314 sec","job_status":"done","queue_duration_s":1.289282,"scheduling_latency_s":1.289303,"redis_calls":13,"redis_duration_s":0.002853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015314,"completed_at":"2026-07-09T06:52:08.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.285840] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"975a0e9baf3061ed78282f1e","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-975a0e9baf3061ed78282f1e: start","job_status":"start","queue_duration_s":1.305883,"scheduling_latency_s":1.305904} gitlab # [ 201.286912] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"975a0e9baf3061ed78282f1e","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-975a0e9baf3061ed78282f1e: done: 0.01492 sec","job_status":"done","queue_duration_s":1.305883,"scheduling_latency_s":1.305904,"redis_calls":13,"redis_duration_s":0.0027949999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01492,"completed_at":"2026-07-09T06:52:08.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.363253] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75021a0320670f9ec3150cf7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75021a0320670f9ec3150cf7: start","job_status":"start","queue_duration_s":1.323471,"scheduling_latency_s":1.323492} gitlab # [ 201.364761] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75021a0320670f9ec3150cf7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75021a0320670f9ec3150cf7: done: 0.021487 sec","job_status":"done","queue_duration_s":1.323471,"scheduling_latency_s":1.323492,"redis_calls":13,"redis_duration_s":0.0035180000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021487,"completed_at":"2026-07-09T06:52:08.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.438315] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beca159aeba03a14c86d9227","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beca159aeba03a14c86d9227: start","job_status":"start","queue_duration_s":1.346304,"scheduling_latency_s":1.346324} gitlab # [ 201.439853] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beca159aeba03a14c86d9227","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beca159aeba03a14c86d9227: done: 0.018149 sec","job_status":"done","queue_duration_s":1.346304,"scheduling_latency_s":1.346324,"redis_calls":13,"redis_duration_s":0.003083,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018149,"completed_at":"2026-07-09T06:52:08.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.506923] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cdde8edaf953770ab2859e5","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cdde8edaf953770ab2859e5: start","job_status":"start","queue_duration_s":1.365985,"scheduling_latency_s":1.366011} gitlab # [ 201.508182] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cdde8edaf953770ab2859e5","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cdde8edaf953770ab2859e5: done: 0.016289 sec","job_status":"done","queue_duration_s":1.365985,"scheduling_latency_s":1.366011,"redis_calls":13,"redis_duration_s":0.003035,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016289,"completed_at":"2026-07-09T06:52:08.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.577067] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98804ba2eeacb1d5ac2bf318","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98804ba2eeacb1d5ac2bf318: start","job_status":"start","queue_duration_s":1.384045,"scheduling_latency_s":1.384066} gitlab # [ 201.578289] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98804ba2eeacb1d5ac2bf318","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98804ba2eeacb1d5ac2bf318: done: 0.014919 sec","job_status":"done","queue_duration_s":1.384045,"scheduling_latency_s":1.384066,"redis_calls":13,"redis_duration_s":0.002725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014919,"completed_at":"2026-07-09T06:52:08.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.654850] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b80ab7b97a1a08d6dabb0fe9","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b80ab7b97a1a08d6dabb0fe9: start","job_status":"start","queue_duration_s":1.400175,"scheduling_latency_s":1.400194} gitlab # [ 201.656188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b80ab7b97a1a08d6dabb0fe9","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b80ab7b97a1a08d6dabb0fe9: done: 0.014894 sec","job_status":"done","queue_duration_s":1.400175,"scheduling_latency_s":1.400194,"redis_calls":13,"redis_duration_s":0.0027429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014894,"completed_at":"2026-07-09T06:52:09.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.730112] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9087bd94653eebe1d391552","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9087bd94653eebe1d391552: start","job_status":"start","queue_duration_s":1.416283,"scheduling_latency_s":1.416303} gitlab # [ 201.731660] gitaly[1397]: time="2026-07-09T06:52:08.107Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:18.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.101" grpc.time_ms=5.831 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.799989] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9087bd94653eebe1d391552","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9087bd94653eebe1d391552: done: 0.018312 sec","job_status":"done","queue_duration_s":1.416283,"scheduling_latency_s":1.416303,"redis_calls":13,"redis_duration_s":0.003098,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018312,"completed_at":"2026-07-09T06:52:09.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.801697] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"933093bb279fb6387de04b88","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-933093bb279fb6387de04b88: start","job_status":"start","queue_duration_s":1.43652,"scheduling_latency_s":1.436547} gitlab # [ 201.875123] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"933093bb279fb6387de04b88","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-933093bb279fb6387de04b88: done: 0.016051 sec","job_status":"done","queue_duration_s":1.43652,"scheduling_latency_s":1.436547,"redis_calls":13,"redis_duration_s":0.002787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016051,"completed_at":"2026-07-09T06:52:09.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.877040] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"754c0d491990cdf4f6ca63c8","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-754c0d491990cdf4f6ca63c8: start","job_status":"start","queue_duration_s":1.453865,"scheduling_latency_s":1.453885} gitlab # [ 201.942928] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"754c0d491990cdf4f6ca63c8","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-754c0d491990cdf4f6ca63c8: done: 0.014912 sec","job_status":"done","queue_duration_s":1.453865,"scheduling_latency_s":1.453885,"redis_calls":13,"redis_duration_s":0.002758,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014912,"completed_at":"2026-07-09T06:52:09.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.015286] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ddb080480db1a8837c0eb89","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ddb080480db1a8837c0eb89: start","job_status":"start","queue_duration_s":1.470038,"scheduling_latency_s":1.470059} gitlab # [ 202.016338] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ddb080480db1a8837c0eb89","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ddb080480db1a8837c0eb89: done: 0.015027 sec","job_status":"done","queue_duration_s":1.470038,"scheduling_latency_s":1.470059,"redis_calls":13,"redis_duration_s":0.0028580000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015027,"completed_at":"2026-07-09T06:52:09.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.093228] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b03437371298ef8f268ea35f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b03437371298ef8f268ea35f: start","job_status":"start","queue_duration_s":1.486492,"scheduling_latency_s":1.486512} gitlab # [ 202.094236] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b03437371298ef8f268ea35f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b03437371298ef8f268ea35f: done: 0.014719 sec","job_status":"done","queue_duration_s":1.486492,"scheduling_latency_s":1.486512,"redis_calls":13,"redis_duration_s":0.002777,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014719,"completed_at":"2026-07-09T06:52:09.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.167192] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6c795629e988f66dfbedb30","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6c795629e988f66dfbedb30: start","job_status":"start","queue_duration_s":1.502444,"scheduling_latency_s":1.502463} gitlab # [ 202.168411] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6c795629e988f66dfbedb30","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6c795629e988f66dfbedb30: done: 0.015045 sec","job_status":"done","queue_duration_s":1.502444,"scheduling_latency_s":1.502463,"redis_calls":13,"redis_duration_s":0.002775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015045,"completed_at":"2026-07-09T06:52:09.116Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.187088] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"459ae322a275d66fb7589019","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-459ae322a275d66fb7589019: start","job_status":"start","queue_duration_s":1.51875,"scheduling_latency_s":1.518817} gitlab # [ 202.255181] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"459ae322a275d66fb7589019","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-459ae322a275d66fb7589019: done: 0.015422 sec","job_status":"done","queue_duration_s":1.51875,"scheduling_latency_s":1.518817,"redis_calls":13,"redis_duration_s":0.003092,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015422,"completed_at":"2026-07-09T06:52:09.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.265863] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"445887ed948df69f63b03f60","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-445887ed948df69f63b03f60: start","job_status":"start","queue_duration_s":1.535449,"scheduling_latency_s":1.535469} gitlab # [ 202.340549] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"445887ed948df69f63b03f60","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-445887ed948df69f63b03f60: done: 0.01505 sec","job_status":"done","queue_duration_s":1.535449,"scheduling_latency_s":1.535469,"redis_calls":13,"redis_duration_s":0.002778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01505,"completed_at":"2026-07-09T06:52:09.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.343972] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e5372203dfda149432201f0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e5372203dfda149432201f0: start","job_status":"start","queue_duration_s":1.551932,"scheduling_latency_s":1.551952} gitlab # [ 202.424289] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e5372203dfda149432201f0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e5372203dfda149432201f0: done: 0.014761 sec","job_status":"done","queue_duration_s":1.551932,"scheduling_latency_s":1.551952,"redis_calls":13,"redis_duration_s":0.002731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014761,"completed_at":"2026-07-09T06:52:09.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.426196] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e54c748368607d92157ecd61","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e54c748368607d92157ecd61: start","job_status":"start","queue_duration_s":1.567971,"scheduling_latency_s":1.56799} gitlab # [ 202.441888] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e54c748368607d92157ecd61","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e54c748368607d92157ecd61: done: 0.015021 sec","job_status":"done","queue_duration_s":1.567971,"scheduling_latency_s":1.56799,"redis_calls":13,"redis_duration_s":0.003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015021,"completed_at":"2026-07-09T06:52:09.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.513236] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0251998987fa90dc41f6064f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0251998987fa90dc41f6064f: start","job_status":"start","queue_duration_s":1.584215,"scheduling_latency_s":1.584235} gitlab # [ 202.522801] gitaly[1397]: time="2026-07-09T06:52:08.139Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-09T06:52:18.050" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.130" grpc.time_ms=7.98 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.592118] gitaly[1397]: time="2026-07-09T06:52:08.173Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:08.173" grpc.time_ms=0.497 pid=1397 span.kind=server system=grpc gitlab # [ 202.593479] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0251998987fa90dc41f6064f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0251998987fa90dc41f6064f: done: 0.015156 sec","job_status":"done","queue_duration_s":1.584215,"scheduling_latency_s":1.584235,"redis_calls":13,"redis_duration_s":0.0028510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015156,"completed_at":"2026-07-09T06:52:09.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.677081] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"833757e6c8595916c8ef43ca","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-833757e6c8595916c8ef43ca: start","job_status":"start","queue_duration_s":1.60063,"scheduling_latency_s":1.600651} gitlab # [ 202.678312] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"833757e6c8595916c8ef43ca","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-833757e6c8595916c8ef43ca: done: 0.01512 sec","job_status":"done","queue_duration_s":1.60063,"scheduling_latency_s":1.600651,"redis_calls":13,"redis_duration_s":0.002817,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01512,"completed_at":"2026-07-09T06:52:09.214Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.747110] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e16af30851382b81bb139eb3","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e16af30851382b81bb139eb3: start","job_status":"start","queue_duration_s":1.617202,"scheduling_latency_s":1.617221} gitlab # [ 202.748334] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e16af30851382b81bb139eb3","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e16af30851382b81bb139eb3: done: 0.014964 sec","job_status":"done","queue_duration_s":1.617202,"scheduling_latency_s":1.617221,"redis_calls":13,"redis_duration_s":0.002805,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014964,"completed_at":"2026-07-09T06:52:09.230Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.819085] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0280c66caf57a8aee69e29b4","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0280c66caf57a8aee69e29b4: start","job_status":"start","queue_duration_s":1.634216,"scheduling_latency_s":1.634255} gitlab # [ 202.820102] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0280c66caf57a8aee69e29b4","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0280c66caf57a8aee69e29b4: done: 0.017811 sec","job_status":"done","queue_duration_s":1.634216,"scheduling_latency_s":1.634255,"redis_calls":13,"redis_duration_s":0.002837,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017811,"completed_at":"2026-07-09T06:52:09.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.891651] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e0f138792b351212a7f8a69","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e0f138792b351212a7f8a69: start","job_status":"start","queue_duration_s":1.65346,"scheduling_latency_s":1.65348} gitlab # [ 202.892694] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e0f138792b351212a7f8a69","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e0f138792b351212a7f8a69: done: 0.014886 sec","job_status":"done","queue_duration_s":1.65346,"scheduling_latency_s":1.65348,"redis_calls":13,"redis_duration_s":0.002697,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014886,"completed_at":"2026-07-09T06:52:09.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 202.960871] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6525d7104232f77532f2b9d5","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6525d7104232f77532f2b9d5: start","job_status":"start","queue_duration_s":1.669587,"scheduling_latency_s":1.669607} gitlab # [ 202.962217] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6525d7104232f77532f2b9d5","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6525d7104232f77532f2b9d5: done: 0.014919 sec","job_status":"done","queue_duration_s":1.669587,"scheduling_latency_s":1.669607,"redis_calls":13,"redis_duration_s":0.002706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001829,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014919,"completed_at":"2026-07-09T06:52:09.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.032947] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc7eadc02550020c274e1b46","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc7eadc02550020c274e1b46: start","job_status":"start","queue_duration_s":1.68576,"scheduling_latency_s":1.685816} gitlab # [ 203.033909] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc7eadc02550020c274e1b46","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc7eadc02550020c274e1b46: done: 0.015045 sec","job_status":"done","queue_duration_s":1.68576,"scheduling_latency_s":1.685816,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015045,"completed_at":"2026-07-09T06:52:09.299Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.102893] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c94b4f6156fe95a7d315888","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c94b4f6156fe95a7d315888: start","job_status":"start","queue_duration_s":1.702237,"scheduling_latency_s":1.702257} gitlab # [ 203.105096] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c94b4f6156fe95a7d315888","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c94b4f6156fe95a7d315888: done: 0.014877 sec","job_status":"done","queue_duration_s":1.702237,"scheduling_latency_s":1.702257,"redis_calls":13,"redis_duration_s":0.00279,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014877,"completed_at":"2026-07-09T06:52:09.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.175743] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af92daccbe040ef4d61efc94","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af92daccbe040ef4d61efc94: start","job_status":"start","queue_duration_s":1.718352,"scheduling_latency_s":1.718373} gitlab # [ 203.184760] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af92daccbe040ef4d61efc94","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af92daccbe040ef4d61efc94: done: 0.014555 sec","job_status":"done","queue_duration_s":1.718352,"scheduling_latency_s":1.718373,"redis_calls":13,"redis_duration_s":0.0028079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014555,"completed_at":"2026-07-09T06:52:09.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.290140] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ce4a9503797e1e2cb45c15","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ce4a9503797e1e2cb45c15: start","job_status":"start","queue_duration_s":1.734146,"scheduling_latency_s":1.734165} gitlab # [ 203.291381] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ce4a9503797e1e2cb45c15","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ce4a9503797e1e2cb45c15: done: 0.01464 sec","job_status":"done","queue_duration_s":1.734146,"scheduling_latency_s":1.734165,"redis_calls":13,"redis_duration_s":0.002819,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01464,"completed_at":"2026-07-09T06:52:09.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.365534] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0ff38f527d47d2867040110","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0ff38f527d47d2867040110: start","job_status":"start","queue_duration_s":1.750061,"scheduling_latency_s":1.750082} gitlab # [ 203.366831] gitaly[1397]: time="2026-07-09T06:52:08.196Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:08.196" grpc.time_ms=0.306 pid=1397 span.kind=server system=grpc gitlab # [ 203.368081] gitaly[1397]: time="2026-07-09T06:52:08.206Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV diskcache=0ea82ed2-32b4-44f2-bbdb-fd0e5e0ab01b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:18.068" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.148" pid=1397 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 203.373179] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0ff38f527d47d2867040110","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0ff38f527d47d2867040110: done: 0.014623 sec","job_status":"done","queue_duration_s":1.750061,"scheduling_latency_s":1.750082,"redis_calls":13,"redis_duration_s":0.002802,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014623,"completed_at":"2026-07-09T06:52:09.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.443157] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28a4e48a249b187f2d7f2de0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28a4e48a249b187f2d7f2de0: start","job_status":"start","queue_duration_s":1.766003,"scheduling_latency_s":1.766023} gitlab # [ 203.512149] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28a4e48a249b187f2d7f2de0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28a4e48a249b187f2d7f2de0: done: 0.015036 sec","job_status":"done","queue_duration_s":1.766003,"scheduling_latency_s":1.766023,"redis_calls":13,"redis_duration_s":0.003006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015036,"completed_at":"2026-07-09T06:52:09.379Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.513640] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6d26199f94105986664017b","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6d26199f94105986664017b: start","job_status":"start","queue_duration_s":1.782288,"scheduling_latency_s":1.782307} gitlab # [ 203.530054] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6d26199f94105986664017b","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6d26199f94105986664017b: done: 0.018477 sec","job_status":"done","queue_duration_s":1.782288,"scheduling_latency_s":1.782307,"redis_calls":13,"redis_duration_s":0.0030830000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018477,"completed_at":"2026-07-09T06:52:09.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.597946] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85fec29dd4cc6607ccfa439d","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85fec29dd4cc6607ccfa439d: start","job_status":"start","queue_duration_s":1.802266,"scheduling_latency_s":1.802288} gitlab # [ 203.608417] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85fec29dd4cc6607ccfa439d","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85fec29dd4cc6607ccfa439d: done: 0.015156 sec","job_status":"done","queue_duration_s":1.802266,"scheduling_latency_s":1.802288,"redis_calls":13,"redis_duration_s":0.0027229999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015156,"completed_at":"2026-07-09T06:52:09.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.679894] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf0f070d5311e4d7b039e02","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf0f070d5311e4d7b039e02: start","job_status":"start","queue_duration_s":1.818632,"scheduling_latency_s":1.818652} gitlab # [ 203.751161] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf0f070d5311e4d7b039e02","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf0f070d5311e4d7b039e02: done: 0.01469 sec","job_status":"done","queue_duration_s":1.818632,"scheduling_latency_s":1.818652,"redis_calls":13,"redis_duration_s":0.002774,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01469,"completed_at":"2026-07-09T06:52:09.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.752896] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92a01a4da994b99da4521ccf","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92a01a4da994b99da4521ccf: start","job_status":"start","queue_duration_s":1.834626,"scheduling_latency_s":1.834646} gitlab # [ 203.823097] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92a01a4da994b99da4521ccf","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92a01a4da994b99da4521ccf: done: 0.014967 sec","job_status":"done","queue_duration_s":1.834626,"scheduling_latency_s":1.834646,"redis_calls":13,"redis_duration_s":0.002859,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014967,"completed_at":"2026-07-09T06:52:09.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.825203] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"305da8dd73a8ca051b9a0675","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-305da8dd73a8ca051b9a0675: start","job_status":"start","queue_duration_s":1.850942,"scheduling_latency_s":1.850962} gitlab # [ 203.893154] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"305da8dd73a8ca051b9a0675","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-305da8dd73a8ca051b9a0675: done: 0.014927 sec","job_status":"done","queue_duration_s":1.850942,"scheduling_latency_s":1.850962,"redis_calls":13,"redis_duration_s":0.0027359999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014927,"completed_at":"2026-07-09T06:52:09.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.902987] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dc60bb96f999377c60434b4","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dc60bb96f999377c60434b4: start","job_status":"start","queue_duration_s":1.867131,"scheduling_latency_s":1.86715} gitlab # [ 203.975309] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dc60bb96f999377c60434b4","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dc60bb96f999377c60434b4: done: 0.015165 sec","job_status":"done","queue_duration_s":1.867131,"scheduling_latency_s":1.86715,"redis_calls":13,"redis_duration_s":0.0029899999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015165,"completed_at":"2026-07-09T06:52:09.481Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 203.981858] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ce0105a1b91f2d402b7bda7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ce0105a1b91f2d402b7bda7: start","job_status":"start","queue_duration_s":1.883544,"scheduling_latency_s":1.883565} gitlab # [ 204.000505] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ce0105a1b91f2d402b7bda7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ce0105a1b91f2d402b7bda7: done: 0.014768 sec","job_status":"done","queue_duration_s":1.883544,"scheduling_latency_s":1.883565,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014768,"completed_at":"2026-07-09T06:52:09.497Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.092959] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"543d423a460785617298f5bd","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-543d423a460785617298f5bd: start","job_status":"start","queue_duration_s":1.899539,"scheduling_latency_s":1.899559} gitlab # [ 204.093963] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"543d423a460785617298f5bd","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-543d423a460785617298f5bd: done: 0.014623 sec","job_status":"done","queue_duration_s":1.899539,"scheduling_latency_s":1.899559,"redis_calls":13,"redis_duration_s":0.002814,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014623,"completed_at":"2026-07-09T06:52:09.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.165052] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bd754168800b3468f2b8930","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bd754168800b3468f2b8930: start","job_status":"start","queue_duration_s":1.915394,"scheduling_latency_s":1.915415} gitlab # [ 204.166729] gitaly[1397]: time="2026-07-09T06:52:08.207Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=58 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4952 command.oublock=8 command.real_time_ms=50 command.spawn_token_fork_ms=0 command.system_time_ms=38 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:18.068" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.148" grpc.time_ms=58.529 pid=1397 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 204.236459] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bd754168800b3468f2b8930","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bd754168800b3468f2b8930: done: 0.014681 sec","job_status":"done","queue_duration_s":1.915394,"scheduling_latency_s":1.915415,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014681,"completed_at":"2026-07-09T06:52:09.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.238711] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1776d243705de3b2ffe43773","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1776d243705de3b2ffe43773: start","job_status":"start","queue_duration_s":1.93131,"scheduling_latency_s":1.931329} gitlab # [ 204.306156] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1776d243705de3b2ffe43773","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1776d243705de3b2ffe43773: done: 0.014983 sec","job_status":"done","queue_duration_s":1.93131,"scheduling_latency_s":1.931329,"redis_calls":13,"redis_duration_s":0.002863,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014983,"completed_at":"2026-07-09T06:52:09.545Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.317048] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ab7b0fd88abd7f971c1436f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ab7b0fd88abd7f971c1436f: start","job_status":"start","queue_duration_s":1.947537,"scheduling_latency_s":1.947558} gitlab # [ 204.336440] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ab7b0fd88abd7f971c1436f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ab7b0fd88abd7f971c1436f: done: 0.017686 sec","job_status":"done","queue_duration_s":1.947537,"scheduling_latency_s":1.947558,"redis_calls":13,"redis_duration_s":0.0032120000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002292,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017686,"completed_at":"2026-07-09T06:52:09.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.438607] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dbd27e6aa2be018c96ad2c6","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dbd27e6aa2be018c96ad2c6: start","job_status":"start","queue_duration_s":1.966722,"scheduling_latency_s":1.966749} gitlab # [ 204.439925] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dbd27e6aa2be018c96ad2c6","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dbd27e6aa2be018c96ad2c6: done: 0.015544 sec","job_status":"done","queue_duration_s":1.966722,"scheduling_latency_s":1.966749,"redis_calls":13,"redis_duration_s":0.002966,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015544,"completed_at":"2026-07-09T06:52:09.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.533113] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.582Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"39326dbfe6bdc1adb746f769","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-39326dbfe6bdc1adb746f769: start","job_status":"start","queue_duration_s":1.642012,"scheduling_latency_s":1.642031} gitlab # [ 204.534350] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.592Z","message":"No existing merge request to be cleaned up."} gitlab # [ 204.534873] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.599Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"39326dbfe6bdc1adb746f769","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-39326dbfe6bdc1adb746f769: done: 0.016748 sec","job_status":"done","queue_duration_s":1.642012,"scheduling_latency_s":1.642031,"redis_calls":10,"redis_duration_s":0.00231,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001422,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016748,"completed_at":"2026-07-09T06:52:09.599Z","load_balancing_strategy":"primary","db_duration_s":0.002262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.616232] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.600Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5fd31ef7f8bde1177be87b00","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5fd31ef7f8bde1177be87b00: start","job_status":"start","queue_duration_s":1.660064,"scheduling_latency_s":1.660084} gitlab # [ 204.617702] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.609Z","message":"No existing merge request to be cleaned up."} gitlab # [ 204.618811] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.615Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5fd31ef7f8bde1177be87b00","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5fd31ef7f8bde1177be87b00: done: 0.015128 sec","job_status":"done","queue_duration_s":1.660064,"scheduling_latency_s":1.660084,"redis_calls":10,"redis_duration_s":0.002193,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001309,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015128,"completed_at":"2026-07-09T06:52:09.615Z","load_balancing_strategy":"primary","db_duration_s":0.001496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.685189] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.616Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e5b7ce3c5a6d0db42c12ae8f","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e5b7ce3c5a6d0db42c12ae8f: start","job_status":"start","queue_duration_s":1.676564,"scheduling_latency_s":1.676584} gitlab # [ 204.686714] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.626Z","message":"No existing merge request to be cleaned up."} gitlab # [ 204.687777] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.632Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e5b7ce3c5a6d0db42c12ae8f","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e5b7ce3c5a6d0db42c12ae8f: done: 0.015321 sec","job_status":"done","queue_duration_s":1.676564,"scheduling_latency_s":1.676584,"redis_calls":10,"redis_duration_s":0.002186,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015321,"completed_at":"2026-07-09T06:52:09.632Z","load_balancing_strategy":"primary","db_duration_s":0.00149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.759128] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.633Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55fb4781e7fa87f8d957dfb0","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55fb4781e7fa87f8d957dfb0: start","job_status":"start","queue_duration_s":1.693511,"scheduling_latency_s":1.693532} gitlab # [ 204.760369] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.643Z","message":"No existing merge request to be cleaned up."} gitlab # [ 204.761210] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.649Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55fb4781e7fa87f8d957dfb0","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55fb4781e7fa87f8d957dfb0: done: 0.015562 sec","job_status":"done","queue_duration_s":1.693511,"scheduling_latency_s":1.693532,"redis_calls":10,"redis_duration_s":0.002247,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001328,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015562,"completed_at":"2026-07-09T06:52:09.649Z","load_balancing_strategy":"primary","db_duration_s":0.001828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.849482] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.650Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"198bc18d10ce8e7cb17ca34b","created_at":"2026-07-09T06:52:08.110Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.114Z","job_size_bytes":46,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-198bc18d10ce8e7cb17ca34b: start","job_status":"start","queue_duration_s":1.536815,"scheduling_latency_s":1.536836} gitlab # [ 204.850684] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.729Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"198bc18d10ce8e7cb17ca34b","created_at":"2026-07-09T06:52:08.110Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.114Z","job_size_bytes":46,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-198bc18d10ce8e7cb17ca34b: done: 0.078945 sec","job_status":"done","queue_duration_s":1.536815,"scheduling_latency_s":1.536836,"gitaly_calls":2,"gitaly_duration_s":0.01412,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007125,"redis_read_bytes":68,"redis_write_bytes":4889,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001296,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000578,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001589,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002664,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340266,"duration_s":0.078945,"completed_at":"2026-07-09T06:52:09.729Z","load_balancing_strategy":"primary","db_duration_s":0.010362,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 204.949986] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.731Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ee8ea92ce7eb940d0d401dac","created_at":"2026-07-09T06:52:08.123Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.127Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ee8ea92ce7eb940d0d401dac: start","job_status":"start","queue_duration_s":1.604516,"scheduling_latency_s":1.604538} gitlab # [ 204.951248] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.808Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ee8ea92ce7eb940d0d401dac","created_at":"2026-07-09T06:52:08.123Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.127Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ee8ea92ce7eb940d0d401dac: done: 0.076464 sec","job_status":"done","queue_duration_s":1.604516,"scheduling_latency_s":1.604538,"gitaly_calls":1,"gitaly_duration_s":0.026591,"redis_calls":15,"redis_duration_s":0.0034460000000000003,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000608,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.054819256999999996,"duration_s":0.076464,"completed_at":"2026-07-09T06:52:09.808Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 205.032059] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.809Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"6d8e988e50b5eca74e2095af","created_at":"2026-07-09T06:52:08.218Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.220Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6d8e988e50b5eca74e2095af: start","job_status":"start","queue_duration_s":1.589656,"scheduling_latency_s":1.589677} gitlab # [ 205.033394] gitaly[1397]: time="2026-07-09T06:52:08.222Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:18.142" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.222" grpc.time_ms=0.22 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.034482] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.846Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"6d8e988e50b5eca74e2095af","created_at":"2026-07-09T06:52:08.218Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.220Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6d8e988e50b5eca74e2095af: done: 0.036723 sec","job_status":"done","queue_duration_s":1.589656,"scheduling_latency_s":1.589677,"gitaly_calls":1,"gitaly_duration_s":0.008379,"redis_calls":16,"redis_duration_s":0.003767,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000267,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001735,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036723,"completed_at":"2026-07-09T06:52:09.846Z","load_balancing_strategy":"primary","db_duration_s":0.002273,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 205.123966] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.847Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2121f21a4cfd0b1f0088fcee","created_at":"2026-07-09T06:52:08.662Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ed5beec3d0aaa88c0a7bcc86d9ed83ae221a756799c3e091b3cbde973ac66c72","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.666Z","job_size_bytes":467,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2121f21a4cfd0b1f0088fcee: start","job_status":"start","queue_duration_s":1.18131,"scheduling_latency_s":1.181332} gitlab # [ 205.125191] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:10.588Z","class":"CreatePipelineWorker","correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"8cb378872b8fe1f7cde149c561f5802ee16ad182","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"8cb378872b8fe1f7cde149c561f5802ee16ad182","ref":"refs/heads/master"},"retry":0} gitlab # [ 205.198079] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.593Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2121f21a4cfd0b1f0088fcee","created_at":"2026-07-09T06:52:08.662Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ed5beec3d0aaa88c0a7bcc86d9ed83ae221a756799c3e091b3cbde973ac66c72","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.666Z","job_size_bytes":467,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2121f21a4cfd0b1f0088fcee: done: 0.745913 sec","job_status":"done","queue_duration_s":1.18131,"scheduling_latency_s":1.181332,"gitaly_calls":7,"gitaly_duration_s":0.052917,"redis_calls":53,"redis_duration_s":0.009913,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.000862,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004541,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002368,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.622462,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000326298,"duration_s":0.745913,"completed_at":"2026-07-09T06:52:10.593Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050387,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 205.273499] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.595Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"1ea2965a9bc3648188c0ce9d","created_at":"2026-07-09T06:52:09.714Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-543eda3ffbc2465e","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:09.719Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-1ea2965a9bc3648188c0ce9d: start","job_status":"start","queue_duration_s":0.876007,"scheduling_latency_s":0.876029} gitlab # [ 205.274624] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.638Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"1ea2965a9bc3648188c0ce9d","created_at":"2026-07-09T06:52:09.714Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-543eda3ffbc2465e","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:09.719Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-1ea2965a9bc3648188c0ce9d: done: 0.043032 sec","job_status":"done","queue_duration_s":0.876007,"scheduling_latency_s":0.876029,"redis_calls":19,"redis_duration_s":0.004334,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00096,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.00064,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001998,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006407797,"duration_s":0.043032,"completed_at":"2026-07-09T06:52:10.638Z","load_balancing_strategy":"primary","db_duration_s":0.005534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 205.305158] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.639Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"c88ce8773b167a5659566ebc","created_at":"2026-07-09T06:52:10.627Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-60d8021c7cc14146","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:10.631Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-c88ce8773b167a5659566ebc: start","job_status":"start","queue_duration_s":0.008241,"scheduling_latency_s":0.008261} gitlab # [ 205.376139] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.701Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"c88ce8773b167a5659566ebc","created_at":"2026-07-09T06:52:10.627Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-60d8021c7cc14146","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:10.631Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-c88ce8773b167a5659566ebc: done: 0.062116 sec","job_status":"done","queue_duration_s":0.008241,"scheduling_latency_s":0.008261,"redis_calls":11,"redis_duration_s":0.00265,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001757,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000331606,"duration_s":0.062116,"completed_at":"2026-07-09T06:52:10.701Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 205.451699] gitaly[1397]: time="2026-07-09T06:52:08.235Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-09T06:52:18.146" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.226" grpc.time_ms=9.316 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.453005] gitaly[1397]: time="2026-07-09T06:52:08.295Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-09T06:52:38.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:08.289" grpc.time_ms=5.255 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.454075] gitaly[1397]: time="2026-07-09T06:52:08.316Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-09T06:52:18.232" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.312" grpc.time_ms=0.203 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.458245] gitaly[1397]: time="2026-07-09T06:52:08.324Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-09T06:52:18.239" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-09T06:52:08.320" grpc.time_ms=4.387 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.536221] gitaly[1397]: time="2026-07-09T06:52:08.343Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:18.256" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.336" grpc.time_ms=6.529 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.537410] gitaly[1397]: time="2026-07-09T06:52:08.353Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:18.266" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.347" grpc.time_ms=6.294 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.537948] gitaly[1397]: time="2026-07-09T06:52:08.371Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-09T06:52:38.161" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:08.361" grpc.time_ms=9.636 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.542145] gitaly[1397]: time="2026-07-09T06:52:08.690Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-09T06:52:18.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:08.689" grpc.time_ms=1.689 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.608192] gitaly[1397]: time="2026-07-09T06:52:09.689Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-09T12:52:09.686" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:09.687" grpc.time_ms=2.355 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.609408] gitaly[1397]: time="2026-07-09T06:52:09.705Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-09T06:52:39.497" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.698" grpc.time_ms=6.787 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.609892] gitaly[1397]: time="2026-07-09T06:52:09.776Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV diskcache=968ea131-e424-49d0-b9b9-ebcee1bc787b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-09T12:52:09.752" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.752" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.612418] gitaly[1397]: time="2026-07-09T06:52:09.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1284 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-09T12:52:09.752" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.752" grpc.time_ms=24.055 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.680237] gitaly[1397]: time="2026-07-09T06:52:09.839Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-09T06:52:39.632" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.832" grpc.time_ms=6.087 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.681809] gitaly[1397]: time="2026-07-09T06:52:09.882Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:39.680" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.881" grpc.time_ms=0.708 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.682864] gitaly[1397]: time="2026-07-09T06:52:09.891Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-09T06:52:19.204" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:09.884" grpc.time_ms=6.755 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.751374] gitaly[1397]: time="2026-07-09T06:52:09.974Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:39.772" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.972" grpc.time_ms=1.719 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.752880] gitaly[1397]: time="2026-07-09T06:52:09.980Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=194 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-09T06:52:39.777" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.977" grpc.time_ms=2.592 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.753686] gitaly[1397]: time="2026-07-09T06:52:10.055Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:20.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:10.048" grpc.time_ms=6.107 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.758189] gitaly[1397]: time="2026-07-09T06:52:10.546Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-09T06:52:40.334" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:10.535" grpc.time_ms=10.747 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.782179] gitaly[1397]: time="2026-07-09T06:52:10.557Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-09T06:52:40.348" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:10.548" grpc.time_ms=9.189 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.128341] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 6549269477 wd_nsec: 6549269448 gitlab # [ 206.817647] nixbot[3109]: {"ts": "2026-07-09T06:52:15.620Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 206.837489] nixbot[3109]: {"ts": "2026-07-09T06:52:15.640Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 206.868697] nixbot[3109]: {"ts": "2026-07-09T06:52:15.671Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 206.875133] nixbot[3109]: {"ts": "2026-07-09T06:52:15.678Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 206.881659] nixbot[3109]: {"ts": "2026-07-09T06:52:15.684Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 206.905775] nixbot[3109]: {"ts": "2026-07-09T06:52:15.708Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 206.912120] nixbot[3109]: {"ts": "2026-07-09T06:52:15.715Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 206.916153] nixbot[3109]: {"ts": "2026-07-09T06:52:15.719Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 206.925735] nixbot[3109]: {"ts": "2026-07-09T06:52:15.728Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 206.929804] nixbot[3109]: {"ts": "2026-07-09T06:52:15.732Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 206.938167] nixbot[3109]: {"ts": "2026-07-09T06:52:15.741Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 206.948197] nixbot[3109]: {"ts": "2026-07-09T06:52:15.751Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 206.954943] nixbot[3109]: {"ts": "2026-07-09T06:52:15.758Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 206.959671] nixbot[3109]: {"ts": "2026-07-09T06:52:15.762Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 206.966653] nixbot[3109]: {"ts": "2026-07-09T06:52:15.769Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 206.970947] nixbot[3109]: {"ts": "2026-07-09T06:52:15.774Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 206.974840] nixbot[3109]: {"ts": "2026-07-09T06:52:15.778Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 206.978834] nixbot[3109]: {"ts": "2026-07-09T06:52:15.782Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 206.986494] nixbot[3109]: {"ts": "2026-07-09T06:52:15.789Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 206.990490] nixbot[3109]: {"ts": "2026-07-09T06:52:15.793Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 206.994642] nixbot[3109]: {"ts": "2026-07-09T06:52:15.797Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 207.325640] nixbot[3109]: INFO: Started server process [3109] gitlab # [ 207.329549] nixbot[3109]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 207.333418] nixbot[3109]: INFO: ::ffff:127.0.0.1:53500 - "GET /health HTTP/1.1" 200 OK gitlab # [ 207.341714] systemd[1]: Started nixbot CI service. gitlab # [ 207.344714] systemd[1]: Reached target Multi-User System. gitlab # [ 207.347979] systemd[1]: Startup finished in 1.609s (kernel) + 4.461s (initrd) + 3min 21.274s (userspace) = 3min 27.344s. gitlab # [ 207.560810] gitaly[1397]: time="2026-07-09T06:52:16.363Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC1C5PZ1JB18F4EZTZK8T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-09T06:52:46.242" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:16.343" grpc.time_ms=19.845 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.662975] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:16 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 331 gitlab # [ 207.665974] nixbot[3109]: {"ts": "2026-07-09T06:52:16.467Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 207.696418] nixbot[3109]: {"ts": "2026-07-09T06:52:16.499Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 207.855153] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 144 gitlab # [ 207.865886] nixbot[3109]: {"ts": "2026-07-09T06:52:16.668Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 207.876710] nixbot[3109]: {"ts": "2026-07-09T06:52:16.679Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 208.015097] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 127 gitlab # [ 208.024511] nixbot[3109]: {"ts": "2026-07-09T06:52:16.827Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 208.166256] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:16 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 119 gitlab # [ 208.168983] nixbot[3109]: {"ts": "2026-07-09T06:52:16.970Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 208.238430] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 44 gitlab # [ 208.241223] nixbot[3109]: {"ts": "2026-07-09T06:52:17.042Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 208.287232] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 44 gitlab # [ 208.289976] nixbot[3109]: {"ts": "2026-07-09T06:52:17.090Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab: (finished: waiting for unit nixbot.service, in 9.29 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 208.406347] gitaly[1397]: time="2026-07-09T06:52:17.209Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2AR2J7TF9Q68MB8E4DJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-09T06:52:47.091" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:17.192" grpc.time_ms=15.918 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.424201] gitaly[1397]: time="2026-07-09T06:52:17.227Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2AR2J7TF9Q68MB8E4DJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:47.124" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:17.225" grpc.time_ms=1.049 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.481916] gitaly[1397]: time="2026-07-09T06:52:17.285Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2AR2J7TF9Q68MB8E4DJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-09T06:52:27.010" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:17.231" grpc.time_ms=53.713 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.496867] nixbot[3109]: INFO: ::ffff:127.0.0.1:53504 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.11 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 208.552928] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 243 gitlab # [ 208.556974] nixbot[3109]: {"ts": "2026-07-09T06:52:17.357Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 208.608625] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 80 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds) (finished: subtest: project discovered and webhook registered, in 0.11 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 208.675500] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 23 gitlab # [ 208.684697] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 125 gitlab # [ 208.695872] nixbot[3109]: {"ts": "2026-07-09T06:52:17.498Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 208.709386] nixbot[3109]: {"ts": "2026-07-09T06:52:17.512Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 208.744275] gitaly[1397]: time="2026-07-09T06:52:17.546Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2PFSABP39MNKSPPT2V4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.537" grpc.time_ms=8.858 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 208.746235] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 61 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 208.833732] gitaly[1397]: time="2026-07-09T06:52:17.631Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2S5X7D3XTS103B7330W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.615" grpc.time_ms=15.053 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 208.835044] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 60 gitlab # [ 208.946376] gitaly[1397]: time="2026-07-09T06:52:17.744Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2V49RXTEEKEE0N1ZRA4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.730" pid=1397 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc gitlab # [ 208.947982] gitlab-workhorse[1306]: time="2026-07-09T06:52:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=149 gitlab # [ 208.949868] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 110 gitlab # [ 208.951105] gitaly[1397]: time="2026-07-09T06:52:17.744Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2V49RXTEEKEE0N1ZRA4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.730" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 208.953049] gitaly[1397]: time="2026-07-09T06:52:17.745Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2V49RXTEEKEE0N1ZRA4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.730" grpc.time_ms=14.336 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.031178] gitaly[1397]: time="2026-07-09T06:52:17.834Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2YRJ9Q48ZRS13X4QS5M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.817" pid=1397 remote_ip="2001:db8:1::1" request_sha=87147561dfef65e1b2b7d6831dfe8d56f471701b response_bytes=229 span.kind=server system=grpc gitlab # [ 209.034889] gitaly[1397]: time="2026-07-09T06:52:17.836Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2YRJ9Q48ZRS13X4QS5M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.817" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.036208] gitlab-workhorse[1306]: time="2026-07-09T06:52:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=229 gitlab # [ 209.036993] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 229 "" "git/2.54.0" 84 gitlab # [ 209.037898] gitaly[1397]: time="2026-07-09T06:52:17.836Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2YRJ9Q48ZRS13X4QS5M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.817" grpc.time_ms=19.069 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.156776] gitaly[1397]: time="2026-07-09T06:52:17.959Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-09T06:52:27.001" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:17.952" grpc.time_ms=7.153 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.184224] gitaly[1397]: time="2026-07-09T06:52:17.987Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=333 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-09T06:52:47.876" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-09T06:52:17.976" grpc.time_ms=11.107 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.197102] gitaly[1397]: time="2026-07-09T06:52:17.999Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC34HQ6D87B4JJFCFZGVG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.980" grpc.time_ms=19.277 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.198084] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 64 gitlab # [ 209.201556] gitaly[1397]: time="2026-07-09T06:52:18.004Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 duration_ms=99ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 209.229567] gitaly[1397]: time="2026-07-09T06:52:18.033Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 duration_ms=27ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 209.231962] gitaly[1397]: time="2026-07-09T06:52:18.035Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:17.904" grpc.time_ms=131.064 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.263615] gitaly[1397]: time="2026-07-09T06:52:18.067Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC36MVQ8YDY1V4C049FBJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.057" pid=1397 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab # [ 209.265074] gitaly[1397]: time="2026-07-09T06:52:18.067Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC36MVQ8YDY1V4C049FBJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.057" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.275629] gitaly[1397]: time="2026-07-09T06:52:18.078Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC36MVQ8YDY1V4C049FBJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.057" grpc.time_ms=21.196 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.277333] gitlab-workhorse[1306]: time="2026-07-09T06:52:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 209.278763] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 75 gitlab # [ 209.314961] gitaly[1397]: time="2026-07-09T06:52:18.118Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:18.116" grpc.time_ms=1.73 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.370108] gitaly[1397]: time="2026-07-09T06:52:18.173Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:18.172" grpc.time_ms=0.56 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.410607] gitaly[1397]: time="2026-07-09T06:52:18.214Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:18.213" grpc.time_ms=0.434 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.443401] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 70 gitlab # [ 209.444841] gitaly[1397]: time="2026-07-09T06:52:18.245Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3BZG692AFAAPCY0EZ94 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.232" grpc.time_ms=12.516 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.491349] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.294Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","8cb378872b8fe1f7cde149c561f5802ee16ad182 4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b7f6af44dfa26baeb21cf232","created_at":"2026-07-09T06:52:18.290Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-ec2b0fb054e04464","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.292Z","job_size_bytes":185,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b7f6af44dfa26baeb21cf232: start","job_status":"start","queue_duration_s":0.001737,"scheduling_latency_s":0.001776} gitlab # [ 209.512686] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.315Z","message":"changes: \"8cb378872b8fe1f7cde149c561f5802ee16ad182 4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf refs/heads/master\\n\""} gitlab # [ 209.519345] gitaly[1397]: time="2026-07-09T06:52:18.322Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3EAKHNFCV6DF786WZS5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.307" pid=1397 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=214 span.kind=server system=grpc gitlab # [ 209.521441] gitaly[1397]: time="2026-07-09T06:52:18.322Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3EAKHNFCV6DF786WZS5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.307" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.522402] gitlab-workhorse[1306]: time="2026-07-09T06:52:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=214 gitlab # [ 209.523701] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 214 "" "git/2.54.0" 73 gitlab # [ 209.525106] gitaly[1397]: time="2026-07-09T06:52:18.323Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3EAKHNFCV6DF786WZS5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.307" grpc.time_ms=16.002 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 209.527092] gitaly[1397]: time="2026-07-09T06:52:18.330Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 duration_ms=64ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 209.529307] gitaly[1397]: time="2026-07-09T06:52:18.332Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:18.265" grpc.time_ms=66.903 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.546249] gitaly[1397]: time="2026-07-09T06:52:18.349Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:28.260" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.341" grpc.time_ms=8.318 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.560663] gitaly[1397]: time="2026-07-09T06:52:18.361Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:28.273" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:18.354" grpc.time_ms=7.518 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.637553] gitaly[1397]: time="2026-07-09T06:52:18.385Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-09T06:52:28.295" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.375" grpc.time_ms=9.717 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.639108] gitaly[1397]: time="2026-07-09T06:52:18.402Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-09T06:52:48.189" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.390" grpc.time_ms=12.169 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.648232] gitaly[1397]: time="2026-07-09T06:52:18.451Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 diskcache=b5e09037-b030-48a4-90ea-c059e1c9ad40 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.699" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.650128] gitaly[1397]: time="2026-07-09T06:52:18.453Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=261 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=12453 command.oublock=32 command.real_time_ms=736 command.spawn_token_fork_ms=2 command.system_time_ms=215 command.user_time_ms=46 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.699" grpc.time_ms=753.438 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.718933] gitlab-workhorse[1306]: time="2026-07-09T06:52:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 209.719661] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 926 gitlab # To http://gitlab/root/test-flake.git gitlab # 8cb3788..4628e24 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.13 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab # [ 209.741851] gitaly[1397]: time="2026-07-09T06:52:18.545Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-09T06:52:28.008" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-09T06:52:18.438" grpc.time_ms=106.504 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf' gitlab # [ 209.764377] gitaly[1397]: time="2026-07-09T06:52:18.567Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:28.125" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.555" grpc.time_ms=10.524 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.811699] nixbot[3109]: INFO: ::ffff:127.0.0.1:53508 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', in 0.05 seconds) gitlab # [ 209.843371] gitaly[1397]: time="2026-07-09T06:52:18.646Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:48.545" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.645" grpc.time_ms=0.659 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.892511] gitaly[1397]: time="2026-07-09T06:52:18.695Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-09T06:52:48.588" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.688" grpc.time_ms=7.092 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.920993] gitaly[1397]: time="2026-07-09T06:52:18.724Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:28.247" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.718" grpc.time_ms=6.105 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.958413] gitaly[1397]: time="2026-07-09T06:52:18.761Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-09T06:52:28.079" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.759" grpc.time_ms=1.683 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.986981] gitaly[1397]: time="2026-07-09T06:52:18.790Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-09T06:52:28.009" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:18.789" grpc.time_ms=0.23 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.999699] gitaly[1397]: time="2026-07-09T06:52:18.803Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=618 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-09T06:52:28.016" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.797" grpc.time_ms=5.823 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.036618] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.839Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","8cb378872b8fe1f7cde149c561f5802ee16ad182 4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b7f6af44dfa26baeb21cf232","created_at":"2026-07-09T06:52:18.290Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-ec2b0fb054e04464","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.292Z","job_size_bytes":185,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b7f6af44dfa26baeb21cf232: done: 0.545449 sec","job_status":"done","queue_duration_s":0.001737,"scheduling_latency_s":0.001776,"gitaly_calls":7,"gitaly_duration_s":0.201288,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.072283,"redis_read_bytes":350,"redis_write_bytes":13068,"redis_cache_calls":9,"redis_cache_duration_s":0.004662,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.020869,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001217,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6768,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003074,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.01204,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.030421,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.025,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.196576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002268445,"exclusive_lock_hold_duration_s":0.022666568,"duration_s":0.545449,"completed_at":"2026-07-09T06:52:18.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035697,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 210.048351] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.844Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1a5ce2f5cb4330a03290f3da","created_at":"2026-07-09T06:52:18.363Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.366Z","job_size_bytes":46,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1a5ce2f5cb4330a03290f3da: start","job_status":"start","queue_duration_s":0.478072,"scheduling_latency_s":0.478104} gitlab # [ 210.125987] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.879Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1a5ce2f5cb4330a03290f3da","created_at":"2026-07-09T06:52:18.363Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.366Z","job_size_bytes":46,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1a5ce2f5cb4330a03290f3da: done: 0.035183 sec","job_status":"done","queue_duration_s":0.478072,"scheduling_latency_s":0.478104,"redis_calls":9,"redis_duration_s":0.0053029999999999996,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000301,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002886,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035183,"completed_at":"2026-07-09T06:52:18.879Z","load_balancing_strategy":"primary","db_duration_s":0.002437,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 210.197159] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.881Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b8c955a3e94fa05cbe47b2c1","created_at":"2026-07-09T06:52:18.370Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.372Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b8c955a3e94fa05cbe47b2c1: start","job_status":"start","queue_duration_s":0.509668,"scheduling_latency_s":0.509695} gitlab # [ 210.198569] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.954Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b8c955a3e94fa05cbe47b2c1","created_at":"2026-07-09T06:52:18.370Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.372Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b8c955a3e94fa05cbe47b2c1: done: 0.073087 sec","job_status":"done","queue_duration_s":0.509668,"scheduling_latency_s":0.509695,"gitaly_calls":1,"gitaly_duration_s":0.038748,"redis_calls":12,"redis_duration_s":0.004096,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000903,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.055107563,"duration_s":0.073087,"completed_at":"2026-07-09T06:52:18.954Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 210.291119] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.956Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f1781817544c4778c19f29df","created_at":"2026-07-09T06:52:18.623Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6d6e9817cc614eeaa9f3f9053a817f1e4d7146d29c31755c8f68eebea7c941d1","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.626Z","job_size_bytes":467,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f1781817544c4778c19f29df: start","job_status":"start","queue_duration_s":0.329554,"scheduling_latency_s":0.329576} gitlab # [ 210.292457] gitaly[1397]: time="2026-07-09T06:52:18.935Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 diskcache=1bb3a2ca-fc1f-4a0f-97ee-cc14d7de48d0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-09T12:52:18.899" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.899" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.293753] gitaly[1397]: time="2026-07-09T06:52:18.935Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1646 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-09T12:52:18.899" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.899" grpc.time_ms=36.176 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.364199] gitaly[1397]: time="2026-07-09T06:52:18.978Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:48.777" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.978" grpc.time_ms=0.575 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.365656] gitaly[1397]: time="2026-07-09T06:52:18.986Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-09T06:52:28.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.980" grpc.time_ms=6.297 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.367124] gitaly[1397]: time="2026-07-09T06:52:19.013Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:49.012" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.013" grpc.time_ms=0.685 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.367993] gitaly[1397]: time="2026-07-09T06:52:19.020Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=321 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-09T06:52:49.016" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.016" grpc.time_ms=3.49 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.384092] gitaly[1397]: time="2026-07-09T06:52:19.165Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:29.118" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:19.158" grpc.time_ms=6.684 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.474624] gitaly[1397]: time="2026-07-09T06:52:19.278Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:19.276" grpc.time_ms=0.913 pid=1397 span.kind=server system=grpc gitlab # [ 210.514290] gitaly[1397]: time="2026-07-09T06:52:19.317Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:19.316" grpc.time_ms=0.774 pid=1397 span.kind=server system=grpc gitlab # [ 210.538762] gitaly[1397]: time="2026-07-09T06:52:19.342Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G diskcache=1fffc81f-cda8-4f4c-9aa6-a00f06243cb6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:28.024" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:18.805" pid=1397 ref=refs/keep-around/8cb378872b8fe1f7cde149c561f5802ee16ad182 remote_ip="2001:db8:1::1" revision=8cb378872b8fe1f7cde149c561f5802ee16ad182 span.kind=server system=grpc username=root gitlab # [ 210.539881] gitaly[1397]: time="2026-07-09T06:52:19.342Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=264 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=5411 command.oublock=8 command.real_time_ms=526 command.spawn_token_fork_ms=0 command.system_time_ms=201 command.user_time_ms=63 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:28.024" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:18.805" grpc.time_ms=537.133 pid=1397 ref=refs/keep-around/8cb378872b8fe1f7cde149c561f5802ee16ad182 remote_ip="2001:db8:1::1" revision=8cb378872b8fe1f7cde149c561f5802ee16ad182 span.kind=server system=grpc username=root gitlab # [ 210.544729] gitaly[1397]: time="2026-07-09T06:52:19.348Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:49.247" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.347" grpc.time_ms=0.422 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.666293] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 875 gitlab # [ 210.671515] nixbot[3109]: {"ts": "2026-07-09T06:52:19.473Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 \"HTTP/1.1 201 Created\""} gitlab # [ 210.697889] nixbot[3109]: {"ts": "2026-07-09T06:52:19.501Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-8cb378872b8f-2dcca477", "argv0": "bwrap"} gitlab # [ 210.805964] systemd[1]: Started Nix Daemon. gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf' gitlab # [ 210.848940] nixbot[3109]: INFO: ::ffff:127.0.0.1:53514 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', in 0.04 seconds) gitlab # [ 210.862291] nix-daemon[3445]: accepted connection from pid 3444, user nixbot gitlab # [ 210.868527] nix-daemon[3445]: accepted connection from pid 3443, user nixbot gitlab # [ 210.879223] gitaly[1397]: time="2026-07-09T06:52:19.682Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=3 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-09T06:52:49.660" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.661" grpc.time_ms=21.137 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.924167] gitaly[1397]: time="2026-07-09T06:52:19.727Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-09T06:52:49.685" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.686" grpc.time_ms=41.128 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.935501] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:19.738Z","class":"CreatePipelineWorker","correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"8cb378872b8fe1f7cde149c561f5802ee16ad182","after":"4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf","ref":"refs/heads/master"},"retry":0} gitlab # [ 210.940288] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.743Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f1781817544c4778c19f29df","created_at":"2026-07-09T06:52:18.623Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6d6e9817cc614eeaa9f3f9053a817f1e4d7146d29c31755c8f68eebea7c941d1","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.626Z","job_size_bytes":467,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f1781817544c4778c19f29df: done: 0.786944 sec","job_status":"done","queue_duration_s":0.329554,"scheduling_latency_s":0.329576,"gitaly_calls":7,"gitaly_duration_s":0.197708,"redis_calls":33,"redis_duration_s":0.011017,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001266,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.007353,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.104,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.101,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.563669,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.786944,"completed_at":"2026-07-09T06:52:19.743Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.103731,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 211.012296] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.745Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"ad58b130-a013-449a-b541-d97ea263950d\"}"],"class":"WebHookWorker","jid":"bb3f4cbc1fee517d06f37724","created_at":"2026-07-09T06:52:18.722Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7b2c9ec3a9499bac32878a43d0ff93018e4fcbf7fdfa13a67501800a2384c11b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.749Z","job_size_bytes":1749,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-bb3f4cbc1fee517d06f37724: start","job_status":"start","queue_duration_s":0.995459,"scheduling_latency_s":0.995484} gitlab # [ 211.103658] nixbot[3109]: INFO: ::ffff:127.0.0.1:53530 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 211.125243] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.928Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"ad58b130-a013-449a-b541-d97ea263950d\"}"],"class":"WebHookWorker","jid":"bb3f4cbc1fee517d06f37724","created_at":"2026-07-09T06:52:18.722Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7b2c9ec3a9499bac32878a43d0ff93018e4fcbf7fdfa13a67501800a2384c11b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.749Z","job_size_bytes":1749,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-bb3f4cbc1fee517d06f37724: done: 0.182999 sec","job_status":"done","queue_duration_s":0.995459,"scheduling_latency_s":0.995484,"redis_calls":19,"redis_duration_s":0.004783,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001339,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.00036,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00193,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001154,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.01138887799999111,"cpu_s":0.063528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384685,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.182999,"completed_at":"2026-07-09T06:52:19.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 211.133431] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.929Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"81a1eedc830794b873face65","created_at":"2026-07-09T06:52:19.353Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.359Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-81a1eedc830794b873face65: start","job_status":"start","queue_duration_s":0.5703,"scheduling_latency_s":0.570323} gitlab # [ 211.209552] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.988Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"81a1eedc830794b873face65","created_at":"2026-07-09T06:52:19.353Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.359Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-81a1eedc830794b873face65: done: 0.058562 sec","job_status":"done","queue_duration_s":0.5703,"scheduling_latency_s":0.570323,"redis_calls":18,"redis_duration_s":0.010326,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004511,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000550628,"duration_s":0.058562,"completed_at":"2026-07-09T06:52:19.988Z","load_balancing_strategy":"primary","db_duration_s":0.014067,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 211.228943] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.990Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f5dd9395c88859d729d759bc","created_at":"2026-07-09T06:52:19.360Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.363Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f5dd9395c88859d729d759bc: start","job_status":"start","queue_duration_s":0.626757,"scheduling_latency_s":0.62678} gitlab # [ 211.303053] gitaly[1397]: time="2026-07-09T06:52:20.103Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5368RAZ399238YQEY1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.092" grpc.time_ms=10.968 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.304627] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 161 gitlab # [ 211.366811] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:20.169Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f5dd9395c88859d729d759bc","created_at":"2026-07-09T06:52:19.360Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.363Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f5dd9395c88859d729d759bc: done: 0.179129 sec","job_status":"done","queue_duration_s":0.626757,"scheduling_latency_s":0.62678,"redis_calls":15,"redis_duration_s":0.005597,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001123,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001788,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000404242,"duration_s":0.179129,"completed_at":"2026-07-09T06:52:20.169Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 211.370472] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:20.173Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"56b6b01ed8fe7f6e2b7b38fe","created_at":"2026-07-09T06:52:19.448Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.454Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56b6b01ed8fe7f6e2b7b38fe: start","job_status":"start","queue_duration_s":0.718926,"scheduling_latency_s":0.71895} gitlab # [ 211.393225] gitaly[1397]: time="2026-07-09T06:52:20.181Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC589NWZTG1VWHVN725T6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.161" pid=1397 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab # [ 211.467833] gitlab-workhorse[1306]: time="2026-07-09T06:52:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 211.469274] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 77 gitlab # [ 211.471973] gitlab-workhorse[1306]: time="2026-07-09T06:52:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=341 gitlab # [ 211.474206] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 341 "" "git/2.54.0" 79 gitlab # [ 211.476437] gitaly[1397]: time="2026-07-09T06:52:20.182Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC589NWZTG1VWHVN725T6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.161" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.477362] gitaly[1397]: time="2026-07-09T06:52:20.182Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC589NWZTG1VWHVN725T6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.161" grpc.time_ms=20.93 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.479743] gitaly[1397]: time="2026-07-09T06:52:20.268Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5AYBMVE56907DXS4S5P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.253" pid=1397 remote_ip="2001:db8:1::1" request_sha=48ab23ff9c165208b0619f9fed6b02fe185097b7 response_bytes=341 span.kind=server system=grpc gitlab # [ 211.480825] gitaly[1397]: time="2026-07-09T06:52:20.268Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1199 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5AYBMVE56907DXS4S5P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.253" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.548989] gitaly[1397]: time="2026-07-09T06:52:20.269Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1199 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5AYBMVE56907DXS4S5P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.253" grpc.time_ms=15.777 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.677676] gitaly[1397]: time="2026-07-09T06:52:20.481Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5JHYHH4SR5NCR8MYJG5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.471" grpc.time_ms=8.511 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.700408] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 69 gitlab # [ 211.763132] gitaly[1397]: time="2026-07-09T06:52:20.566Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5MVA50QBCBYKDYA35ST grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.548" pid=1397 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 211.767438] gitaly[1397]: time="2026-07-09T06:52:20.568Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5MVA50QBCBYKDYA35ST grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.548" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.768748] gitlab-workhorse[1306]: time="2026-07-09T06:52:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 211.769810] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.54.0" 61 gitlab # [ 211.770806] gitaly[1397]: time="2026-07-09T06:52:20.568Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5MVA50QBCBYKDYA35ST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.548" grpc.time_ms=20.485 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf' gitlab # [ 211.845496] gitaly[1397]: time="2026-07-09T06:52:20.648Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:50.646" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.646" grpc.time_ms=1.79 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.884701] nixbot[3109]: {"ts": "2026-07-09T06:52:20.687Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', in 0.04 seconds) gitlab # [ 211.891067] nixbot[3109]: INFO: ::ffff:127.0.0.1:53538 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 200 OK (finished: subtest: push delivers the webhook, in 3.28 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses gitlab # [ 211.892495] nix-daemon[3454]: unexpected Nix daemon error: error: interrupted by the user gitlab # [ 211.919096] gitaly[1397]: time="2026-07-09T06:52:20.722Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.720" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.721" grpc.time_ms=0.841 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.939058] gitaly[1397]: time="2026-07-09T06:52:20.742Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.641" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.741" grpc.time_ms=0.483 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.964679] gitaly[1397]: time="2026-07-09T06:52:20.768Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5VKS0HGPEK3CJEDCR73 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.666" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.767" grpc.time_ms=0.621 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.975780] gitaly[1397]: time="2026-07-09T06:52:20.779Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-09T06:52:50.671" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:20.772" grpc.time_ms=6.715 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.989332] gitaly[1397]: time="2026-07-09T06:52:20.792Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:30.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:20.785" grpc.time_ms=7.55 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.009381] gitaly[1397]: time="2026-07-09T06:52:20.812Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-09T06:52:30.030" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:20.811" grpc.time_ms=0.199 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.020748] gitaly[1397]: time="2026-07-09T06:52:20.824Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=617 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-09T06:52:30.034" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:20.814" grpc.time_ms=9.172 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.067328] gitaly[1397]: time="2026-07-09T06:52:20.869Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:20.866" grpc.time_ms=2.87 pid=1397 span.kind=server system=grpc gitlab # [ 212.091136] gitaly[1397]: time="2026-07-09T06:52:20.894Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.793" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.893" grpc.time_ms=0.731 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.093770] gitaly[1397]: time="2026-07-09T06:52:20.897Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:20.896" grpc.time_ms=0.277 pid=1397 span.kind=server system=grpc gitlab # [ 212.136235] gitaly[1397]: time="2026-07-09T06:52:20.939Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ diskcache=80b6de0b-76dd-462f-ab07-13eb97b1465e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:30.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:20.828" pid=1397 ref=refs/keep-around/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf remote_ip="2001:db8:1::1" revision=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf span.kind=server system=grpc username=root gitlab # [ 212.154203] gitaly[1397]: time="2026-07-09T06:52:20.957Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=73 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5030 command.oublock=8 command.real_time_ms=100 command.spawn_token_fork_ms=2 command.system_time_ms=49 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:30.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:20.828" grpc.time_ms=111.467 pid=1397 ref=refs/keep-around/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf remote_ip="2001:db8:1::1" revision=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf span.kind=server system=grpc username=root gitlab # [ 212.161090] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 240 gitlab # [ 212.182401] gitaly[1397]: time="2026-07-09T06:52:20.985Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.884" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.985" grpc.time_ms=0.455 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.32 seconds) gitlab # [ 212.204905] gitaly[1397]: time="2026-07-09T06:52:21.008Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.105" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:21.006" grpc.time_ms=2.027 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.231312] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.032Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"56b6b01ed8fe7f6e2b7b38fe","created_at":"2026-07-09T06:52:19.448Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.454Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56b6b01ed8fe7f6e2b7b38fe: done: 0.858708 sec","job_status":"done","queue_duration_s":0.718926,"scheduling_latency_s":0.71895,"gitaly_calls":3,"gitaly_duration_s":0.051139,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.073505,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002751,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004807,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.015237,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000311,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.010742,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.004873,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000539,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.031494,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.454,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.056,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.1,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.196,"db_ci_duration_s":0.259,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.13924,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.737992398,"duration_s":0.858708,"completed_at":"2026-07-09T06:52:21.032Z","load_balancing_strategy":"primary","db_duration_s":0.578757,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 212.313316] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.040Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","bb3f4cbc1fee517d06f37724"],"class":"WebHooks::LogExecutionWorker","jid":"ae5a73aa80952cab17e1e693","created_at":"2026-07-09T06:52:19.911Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-20a2085d91e84699","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2bb16ab6831798fa084469ad062fc4ad8ac966dcd5e66a03480ba1db33bc4720","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.919Z","job_size_bytes":2506,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ae5a73aa80952cab17e1e693: start","job_status":"start","queue_duration_s":1.120448,"scheduling_latency_s":1.120486} gitlab # [ 212.339383] gitaly[1397]: time="2026-07-09T06:52:21.037Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:31.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:21.028" grpc.time_ms=8.952 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.361497] nixbot[3109]: {"ts": "2026-07-09T06:52:21.135Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 \"HTTP/1.1 201 Created\""} gitlab # [ 212.363218] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 316 gitlab # [ 212.368821] nixbot[3109]: {"ts": "2026-07-09T06:52:21.170Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf \"HTTP/1.1 201 Created\""} gitlab # [ 212.369657] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 480 gitlab # [ 212.375068] nixbot[3109]: {"ts": "2026-07-09T06:52:21.178Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-4628e241e0e4-1418f780", "argv0": "bwrap"} gitlab # [ 212.409782] gitaly[1397]: time="2026-07-09T06:52:21.213Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC68M9Q9BC6NY779HQ8EY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:51.110" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:21.211" grpc.time_ms=1.102 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.458478] nix-daemon[3445]: accepted connection from pid 3550, user nixbot gitlab # [ 212.463717] nix-daemon[3445]: accepted connection from pid 3552, user nixbot gitlab # [ 212.481260] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.283Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","bb3f4cbc1fee517d06f37724"],"class":"WebHooks::LogExecutionWorker","jid":"ae5a73aa80952cab17e1e693","created_at":"2026-07-09T06:52:19.911Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-20a2085d91e84699","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2bb16ab6831798fa084469ad062fc4ad8ac966dcd5e66a03480ba1db33bc4720","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.919Z","job_size_bytes":2506,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ae5a73aa80952cab17e1e693: done: 0.24305 sec","job_status":"done","queue_duration_s":1.120448,"scheduling_latency_s":1.120486,"redis_calls":10,"redis_duration_s":0.019931,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001862,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.051,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.051,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00591388,"duration_s":0.24305,"completed_at":"2026-07-09T06:52:21.283Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 212.555280] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.358Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8e5242ea24f29ded20f4c475","created_at":"2026-07-09T06:52:20.617Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-50011061ea3b40bd","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.624Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8e5242ea24f29ded20f4c475: start","job_status":"start","queue_duration_s":0.73313,"scheduling_latency_s":0.733199} gitlab # [ 212.581247] gitaly[1397]: time="2026-07-09T06:52:21.382Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC68M9Q9BC6NY779HQ8EY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:31.209" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:21.371" grpc.time_ms=10.527 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.654156] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 316 gitlab # [ 212.656482] nixbot[3109]: {"ts": "2026-07-09T06:52:21.459Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 \"HTTP/1.1 201 Created\""} gitlab # [ 212.691333] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.494Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8e5242ea24f29ded20f4c475","created_at":"2026-07-09T06:52:20.617Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-50011061ea3b40bd","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.624Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8e5242ea24f29ded20f4c475: done: 0.136607 sec","job_status":"done","queue_duration_s":0.73313,"scheduling_latency_s":0.733199,"redis_calls":10,"redis_duration_s":0.02997,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001966,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.026785,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001219,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001347657,"duration_s":0.136607,"completed_at":"2026-07-09T06:52:21.494Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 212.779970] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.582Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"83de3b3545495bddbf4482f6","created_at":"2026-07-09T06:52:20.993Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.997Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-83de3b3545495bddbf4482f6: start","job_status":"start","queue_duration_s":0.584686,"scheduling_latency_s":0.584761} gitlab # [ 212.889243] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.691Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"83de3b3545495bddbf4482f6","created_at":"2026-07-09T06:52:20.993Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.997Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-83de3b3545495bddbf4482f6: done: 0.109372 sec","job_status":"done","queue_duration_s":0.584686,"scheduling_latency_s":0.584761,"redis_calls":7,"redis_duration_s":0.009635000000000001,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00168,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.109372,"completed_at":"2026-07-09T06:52:21.691Z","load_balancing_strategy":"primary","db_duration_s":0.010842,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 212.976500] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.779Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b800a8f25e64d0e79d3118a8","created_at":"2026-07-09T06:52:20.998Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.999Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b800a8f25e64d0e79d3118a8: start","job_status":"start","queue_duration_s":0.778949,"scheduling_latency_s":0.779027} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses gitlab # [ 213.158186] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.960Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b800a8f25e64d0e79d3118a8","created_at":"2026-07-09T06:52:20.998Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.999Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b800a8f25e64d0e79d3118a8: done: 0.181652 sec","job_status":"done","queue_duration_s":0.778949,"scheduling_latency_s":0.779027,"redis_calls":10,"redis_duration_s":0.024965,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.01244,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.181652,"completed_at":"2026-07-09T06:52:21.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.268804] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:22.071Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"54f80a7955d6d78610868d48","created_at":"2026-07-09T06:52:21.110Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-b06a4f9716da4413","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.115Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-54f80a7955d6d78610868d48: start","job_status":"start","queue_duration_s":0.956056,"scheduling_latency_s":0.956122} gitlab # [ 213.350777] gitaly[1397]: time="2026-07-09T06:52:22.153Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC75DT7FV2EDZZG7MP4RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:52.049" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:22.150" grpc.time_ms=2.586 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.375353] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 116 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.17 seconds) gitlab # [ 213.479405] gitaly[1397]: time="2026-07-09T06:52:22.282Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:22.282" grpc.time_ms=0.481 pid=1397 span.kind=server system=grpc gitlab # [ 213.517474] gitaly[1397]: time="2026-07-09T06:52:22.320Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:22.320" grpc.time_ms=0.458 pid=1397 span.kind=server system=grpc gitlab # [ 213.556861] gitaly[1397]: time="2026-07-09T06:52:22.360Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:22.359" grpc.time_ms=0.499 pid=1397 span.kind=server system=grpc gitlab # [ 213.583923] gitaly[1397]: time="2026-07-09T06:52:22.387Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T diskcache=922b4e1f-2fb7-48be-95bc-5d6c1a95f090 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-09T06:52:52.249" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:22.249" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.585237] gitaly[1397]: time="2026-07-09T06:52:22.387Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=95 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7046 command.oublock=0 command.real_time_ms=127 command.spawn_token_fork_ms=2 command.system_time_ms=53 command.user_time_ms=42 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-09T06:52:52.249" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:22.249" grpc.time_ms=137.774 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.710426] gitaly[1397]: time="2026-07-09T06:52:22.513Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:52.512" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:22.512" grpc.time_ms=1.326 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.800182] gitaly[1397]: time="2026-07-09T06:52:22.603Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:52.600" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:22.601" grpc.time_ms=1.192 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.908582] gitaly[1397]: time="2026-07-09T06:52:22.711Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:52.708" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:22.709" grpc.time_ms=1.58 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.935770] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:22.738Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"54f80a7955d6d78610868d48","created_at":"2026-07-09T06:52:21.110Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-b06a4f9716da4413","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.115Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2147,"job_status":"rescheduled","message":"PipelineProcessWorker JID-54f80a7955d6d78610868d48: rescheduled"} gitlab # [ 213.969436] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:22.771Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"54f80a7955d6d78610868d48","created_at":"2026-07-09T06:52:21.110Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-b06a4f9716da4413","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.115Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-54f80a7955d6d78610868d48: done: 0.700455 sec","job_status":"done","queue_duration_s":0.956056,"scheduling_latency_s":0.956122,"gitaly_calls":4,"gitaly_duration_s":0.188707,"redis_calls":101,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.10816,"redis_read_bytes":185,"redis_write_bytes":23557,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.003765,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.007413,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.02942,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.016325,"redis_queues_read_bytes":25,"redis_queues_write_bytes":12389,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.030963,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00174,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002657,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012112,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.115,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.025,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.056,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.257353,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.58126382,"duration_s":0.700455,"completed_at":"2026-07-09T06:52:22.771Z","load_balancing_strategy":"primary","db_duration_s":0.125395,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 214.046694] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:22.785Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"93d44d0f3a0b7baf47f8071e","created_at":"2026-07-09T06:52:21.150Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.153Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-93d44d0f3a0b7baf47f8071e: start","job_status":"start","queue_duration_s":1.631303,"scheduling_latency_s":1.631367} gitlab # [ 214.227209] gitaly[1397]: time="2026-07-09T06:52:23.030Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:53.027" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:23.028" grpc.time_ms=1.685 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.318196] gitaly[1397]: time="2026-07-09T06:52:23.121Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:53.118" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:23.119" grpc.time_ms=1.484 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses gitlab # [ 214.402257] gitaly[1397]: time="2026-07-09T06:52:23.204Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:53.201" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:23.202" grpc.time_ms=1.554 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.473119] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:23.274Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"93d44d0f3a0b7baf47f8071e","created_at":"2026-07-09T06:52:21.150Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.153Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-93d44d0f3a0b7baf47f8071e: done: 0.489425 sec","job_status":"done","queue_duration_s":1.631303,"scheduling_latency_s":1.631367,"gitaly_calls":3,"gitaly_duration_s":0.028879,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.08449000000000001,"redis_read_bytes":99,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.006994,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.013934,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.004739,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.014397,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001145,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00219,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.034097,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.09,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.055,"db_ci_duration_s":0.035,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.251242,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.304014337,"duration_s":0.489425,"completed_at":"2026-07-09T06:52:23.274Z","load_balancing_strategy":"primary","db_duration_s":0.098825,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 214.559421] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:23.362Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"696a190f7bfc5d3b5a95c5ff","created_at":"2026-07-09T06:52:22.224Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.232Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-696a190f7bfc5d3b5a95c5ff: start","job_status":"start","queue_duration_s":1.129919,"scheduling_latency_s":1.129981} gitlab # [ 214.577440] gitaly[1397]: time="2026-07-09T06:52:23.379Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC8AV87B5KZK1VDBZ09Q8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:53.276" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:23.377" grpc.time_ms=2.272 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.617334] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 160 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.24 seconds) gitlab # [ 214.879191] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:23.681Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"696a190f7bfc5d3b5a95c5ff","created_at":"2026-07-09T06:52:22.224Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.232Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-696a190f7bfc5d3b5a95c5ff: done: 0.319939 sec","job_status":"done","queue_duration_s":1.129919,"scheduling_latency_s":1.129981,"redis_calls":15,"redis_duration_s":0.033391000000000004,"redis_read_bytes":13,"redis_write_bytes":3011,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.002834,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1626,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.016042,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.004345,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005737,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140465,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002698108,"duration_s":0.319939,"completed_at":"2026-07-09T06:52:23.681Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056747,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 214.963302] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:23.765Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"b539a2a4dc2247b662af9bf7","created_at":"2026-07-09T06:52:23.655Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-712dbd74b7e3446b","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:d9e4da9715e14a7006fd5e12e0b6b729123be290a75bfd00943e204e64372d4b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:23.662Z","job_size_bytes":464,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-b539a2a4dc2247b662af9bf7: start","job_status":"start","queue_duration_s":0.103321,"scheduling_latency_s":0.103359} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses gitlab # [ 215.145443] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:23.947Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"b539a2a4dc2247b662af9bf7","created_at":"2026-07-09T06:52:23.655Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-712dbd74b7e3446b","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:d9e4da9715e14a7006fd5e12e0b6b729123be290a75bfd00943e204e64372d4b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:23.662Z","job_size_bytes":464,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-b539a2a4dc2247b662af9bf7: fail: 0.180573 sec","job_status":"fail","queue_duration_s":0.103321,"scheduling_latency_s":0.103359,"redis_calls":7,"redis_duration_s":0.016118,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004244,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.180573,"completed_at":"2026-07-09T06:52:23.946Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.036814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 215.773373] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.575Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1ba10ec184a8a29b7fcac598","created_at":"2026-07-09T06:52:22.395Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.403Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1ba10ec184a8a29b7fcac598: start","job_status":"start","queue_duration_s":2.17162,"scheduling_latency_s":2.171657} gitlab # [ 215.823150] gitaly[1397]: time="2026-07-09T06:52:24.626Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC9KNXCDFCM5FBP8YJXSF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:54.523" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:24.625" grpc.time_ms=0.729 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.854160] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 91 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.24 seconds) gitlab # [ 215.880765] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.683Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1ba10ec184a8a29b7fcac598","created_at":"2026-07-09T06:52:22.395Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.403Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1ba10ec184a8a29b7fcac598: done: 0.108496 sec","job_status":"done","queue_duration_s":2.17162,"scheduling_latency_s":2.171657,"redis_calls":10,"redis_duration_s":0.017474,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005353,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001301562,"duration_s":0.108496,"completed_at":"2026-07-09T06:52:24.683Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012997,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 215.916965] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.688Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7d867453a43cfca4be16bf11","created_at":"2026-07-09T06:52:22.420Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.436Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7d867453a43cfca4be16bf11: start","job_status":"start","queue_duration_s":2.251899,"scheduling_latency_s":2.251931} gitlab # [ 216.035736] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.838Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7d867453a43cfca4be16bf11","created_at":"2026-07-09T06:52:22.420Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.436Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7d867453a43cfca4be16bf11: done: 0.149629 sec","job_status":"done","queue_duration_s":2.251899,"scheduling_latency_s":2.251931,"redis_calls":10,"redis_duration_s":0.019827,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002875,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014873,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002079,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.075,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.072,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003090337,"duration_s":0.149629,"completed_at":"2026-07-09T06:52:24.837Z","load_balancing_strategy":"primary","db_duration_s":0.074475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 216.110408] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.849Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"996009808a53ca940b216e2e","created_at":"2026-07-09T06:52:22.450Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.459Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-996009808a53ca940b216e2e: start","job_status":"start","queue_duration_s":2.390456,"scheduling_latency_s":2.390517} gitlab # [ 216.174828] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.977Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"996009808a53ca940b216e2e","created_at":"2026-07-09T06:52:22.450Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.459Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-996009808a53ca940b216e2e: done: 0.127656 sec","job_status":"done","queue_duration_s":2.390456,"scheduling_latency_s":2.390517,"redis_calls":13,"redis_duration_s":0.040022,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004233,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.020422,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003181689,"duration_s":0.127656,"completed_at":"2026-07-09T06:52:24.977Z","load_balancing_strategy":"primary","db_duration_s":0.01617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 216.211759] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.988Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"dff925da3166ffc0bf985349","created_at":"2026-07-09T06:52:22.460Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.468Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-dff925da3166ffc0bf985349: start","job_status":"start","queue_duration_s":2.519201,"scheduling_latency_s":2.519271} gitlab # [ 216.353769] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.156Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"dff925da3166ffc0bf985349","created_at":"2026-07-09T06:52:22.460Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.468Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-dff925da3166ffc0bf985349: done: 0.168359 sec","job_status":"done","queue_duration_s":2.519201,"scheduling_latency_s":2.519271,"redis_calls":10,"redis_duration_s":0.084706,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004333,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014567,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.065806,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003178058,"duration_s":0.168359,"completed_at":"2026-07-09T06:52:25.156Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 216.456640] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.259Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ab9fad311132c5cba097f5e5","created_at":"2026-07-09T06:52:22.469Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.479Z","job_size_bytes":19,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ab9fad311132c5cba097f5e5: start","job_status":"start","queue_duration_s":2.780167,"scheduling_latency_s":2.78024} gitlab # [ 216.559773] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.362Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ab9fad311132c5cba097f5e5","created_at":"2026-07-09T06:52:22.469Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.479Z","job_size_bytes":19,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ab9fad311132c5cba097f5e5: done: 0.103062 sec","job_status":"done","queue_duration_s":2.780167,"scheduling_latency_s":2.78024,"redis_calls":11,"redis_duration_s":0.024413,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003953,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.016613,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003847,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001464991,"duration_s":0.103062,"completed_at":"2026-07-09T06:52:25.362Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 216.659421] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.461Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c72e6dbbeb1b5ba4e689c873","created_at":"2026-07-09T06:52:22.482Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.492Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c72e6dbbeb1b5ba4e689c873: start","job_status":"start","queue_duration_s":2.969429,"scheduling_latency_s":2.969503} gitlab # [ 216.756725] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.559Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c72e6dbbeb1b5ba4e689c873","created_at":"2026-07-09T06:52:22.482Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.492Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c72e6dbbeb1b5ba4e689c873: done: 0.097646 sec","job_status":"done","queue_duration_s":2.969429,"scheduling_latency_s":2.969503,"redis_calls":10,"redis_duration_s":0.021482,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004112,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002241,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003429486,"duration_s":0.097646,"completed_at":"2026-07-09T06:52:25.559Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses gitlab # [ 216.855407] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.657Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c9a3ccff2cc3416214eef96c","created_at":"2026-07-09T06:52:22.496Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.501Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c9a3ccff2cc3416214eef96c: start","job_status":"start","queue_duration_s":3.155874,"scheduling_latency_s":3.15595} gitlab # [ 216.959263] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.762Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c9a3ccff2cc3416214eef96c","created_at":"2026-07-09T06:52:22.496Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.501Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c9a3ccff2cc3416214eef96c: done: 0.105471 sec","job_status":"done","queue_duration_s":3.155874,"scheduling_latency_s":3.15595,"redis_calls":7,"redis_duration_s":0.013316,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.105471,"completed_at":"2026-07-09T06:52:25.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 216.995692] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.772Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"180b8170d620fdc924a1aae5","created_at":"2026-07-09T06:52:22.607Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.627Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-180b8170d620fdc924a1aae5: start","job_status":"start","queue_duration_s":3.145014,"scheduling_latency_s":3.145075} gitlab # [ 216.997382] gitaly[1397]: time="2026-07-09T06:52:25.799Z" level=info msg="finished unary call with code OK" catfile.duration_ms=13 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=13 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCAQQQMPTAGVTE38FM2VC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:55.682" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:25.783" grpc.time_ms=14.821 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.064706] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 101 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.21 seconds) gitlab # [ 217.142597] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.945Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"180b8170d620fdc924a1aae5","created_at":"2026-07-09T06:52:22.607Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.627Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-180b8170d620fdc924a1aae5: done: 0.172994 sec","job_status":"done","queue_duration_s":3.145014,"scheduling_latency_s":3.145075,"redis_calls":10,"redis_duration_s":0.01807,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002882,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004298,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002856787,"duration_s":0.172994,"completed_at":"2026-07-09T06:52:25.945Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 217.228139] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.030Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a5bd3c1a0ba23b6d5b0c64ef","created_at":"2026-07-09T06:52:22.631Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.646Z","job_size_bytes":6,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a5bd3c1a0ba23b6d5b0c64ef: start","job_status":"start","queue_duration_s":3.384512,"scheduling_latency_s":3.384546} gitlab # [ 217.307439] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.110Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a5bd3c1a0ba23b6d5b0c64ef","created_at":"2026-07-09T06:52:22.631Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.646Z","job_size_bytes":6,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a5bd3c1a0ba23b6d5b0c64ef: done: 0.079805 sec","job_status":"done","queue_duration_s":3.384512,"scheduling_latency_s":3.384546,"redis_calls":10,"redis_duration_s":0.024026000000000002,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001902,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.020039,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000774959,"duration_s":0.079805,"completed_at":"2026-07-09T06:52:26.110Z","load_balancing_strategy":"primary","db_duration_s":0.005296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 217.375542] nix-daemon[3445]: accepted connection from pid 3617, user nixbot gitlab # [ 217.377351] gitaly[1397]: time="2026-07-09T06:52:26.178Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:56.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.140" grpc.time_ms=0.717 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.392416] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.195Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8d73aa45c91e22206fc654be","created_at":"2026-07-09T06:52:22.742Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.750Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8d73aa45c91e22206fc654be: start","job_status":"start","queue_duration_s":3.445271,"scheduling_latency_s":3.445324} gitlab # [ 217.413791] gitaly[1397]: time="2026-07-09T06:52:26.217Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:36.184" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:26.205" grpc.time_ms=11.66 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.467295] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 172 gitlab # [ 217.468471] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.270Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8d73aa45c91e22206fc654be","created_at":"2026-07-09T06:52:22.742Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.750Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8d73aa45c91e22206fc654be: done: 0.075278 sec","job_status":"done","queue_duration_s":3.445271,"scheduling_latency_s":3.445324,"redis_calls":8,"redis_duration_s":0.009764,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006187,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003577,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075278,"completed_at":"2026-07-09T06:52:26.270Z","load_balancing_strategy":"primary","db_duration_s":0.026306,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 217.474922] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"e7c61d1723f7a7f6772ac5be","created_at":"2026-07-09T06:52:22.989Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-c9abab287cf744ab","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.996Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e7c61d1723f7a7f6772ac5be: start","job_status":"start","queue_duration_s":3.282153,"scheduling_latency_s":3.28218} gitlab # [ 217.540344] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"e7c61d1723f7a7f6772ac5be","created_at":"2026-07-09T06:52:22.989Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-c9abab287cf744ab","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.996Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e7c61d1723f7a7f6772ac5be: done: 0.065134 sec","job_status":"done","queue_duration_s":3.282153,"scheduling_latency_s":3.28218,"redis_calls":7,"redis_duration_s":0.006257,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004541,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001716,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065134,"completed_at":"2026-07-09T06:52:26.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 217.613346] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.345Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"72ea4ae898810bd7fbb3b43c","created_at":"2026-07-09T06:52:26.250Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-24ccff0da2fe4245","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.255Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-72ea4ae898810bd7fbb3b43c: start","job_status":"start","queue_duration_s":0.090428,"scheduling_latency_s":0.090456} gitlab # [ 217.641143] gitaly[1397]: time="2026-07-09T06:52:26.444Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:26.442" grpc.time_ms=1.96 pid=1397 span.kind=server system=grpc gitlab # [ 217.664578] gitaly[1397]: time="2026-07-09T06:52:26.468Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:26.467" grpc.time_ms=0.352 pid=1397 span.kind=server system=grpc gitlab # [ 217.666518] nixbot[3109]: {"ts": "2026-07-09T06:52:26.469Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf \"HTTP/1.1 201 Created\""} gitlab # [ 217.691617] gitaly[1397]: time="2026-07-09T06:52:26.495Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:26.494" grpc.time_ms=0.311 pid=1397 span.kind=server system=grpc gitlab # [ 217.713477] gitaly[1397]: time="2026-07-09T06:52:26.516Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC diskcache=06c49cb2-1076-4062-9ced-8003f0f2dc41 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-09T06:52:56.417" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:26.418" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.714977] gitaly[1397]: time="2026-07-09T06:52:26.517Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=83 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6967 command.oublock=0 command.real_time_ms=80 command.spawn_token_fork_ms=0 command.system_time_ms=54 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-09T06:52:56.417" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:26.418" grpc.time_ms=98.62 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.718890] gitaly[1397]: time="2026-07-09T06:52:26.521Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCBF9J846FQDY1V70FB39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:56.419" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.520" grpc.time_ms=0.617 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.762101] gitaly[1397]: time="2026-07-09T06:52:26.565Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:56.564" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.564" grpc.time_ms=1.056 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.784644] gitaly[1397]: time="2026-07-09T06:52:26.588Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:56.587" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.587" grpc.time_ms=0.628 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.813728] gitaly[1397]: time="2026-07-09T06:52:26.616Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCBF9J846FQDY1V70FB39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:36.138" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:26.608" grpc.time_ms=7.746 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.854092] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 183 gitlab # [ 217.855994] nixbot[3109]: {"ts": "2026-07-09T06:52:26.659Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf \"HTTP/1.1 201 Created\""} gitlab # [ 218.018774] gitaly[1397]: time="2026-07-09T06:52:26.821Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:56.819" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.819" grpc.time_ms=1.071 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses gitlab # [ 218.030588] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.832Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"24bbd51d3a97cfa19b8aec54","created_at":"2026-07-09T06:52:26.830Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"72ea4ae898810bd7fbb3b43c","job_size_bytes":3,"pid":2147,"job_status":"deduplicated","message":"PipelineProcessWorker JID-24bbd51d3a97cfa19b8aec54: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 218.034346] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.835Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"72ea4ae898810bd7fbb3b43c","created_at":"2026-07-09T06:52:26.250Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-24ccff0da2fe4245","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.255Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2147,"job_status":"rescheduled","message":"PipelineProcessWorker JID-72ea4ae898810bd7fbb3b43c: rescheduled"} gitlab # [ 218.039415] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.842Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"72ea4ae898810bd7fbb3b43c","created_at":"2026-07-09T06:52:26.250Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-24ccff0da2fe4245","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.255Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-72ea4ae898810bd7fbb3b43c: done: 0.497032 sec","job_status":"done","queue_duration_s":0.090428,"scheduling_latency_s":0.090456,"gitaly_calls":4,"gitaly_duration_s":0.114991,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.025389,"redis_read_bytes":262,"redis_write_bytes":23388,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001119,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.00325,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000945,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.004948,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14469,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.009214,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000252,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001911,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002631,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.214,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.199,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.144943,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.455183829,"duration_s":0.497032,"completed_at":"2026-07-09T06:52:26.842Z","load_balancing_strategy":"primary","db_duration_s":0.217451,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 218.143687] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5ecbfb30e120439b199cb644","created_at":"2026-07-09T06:52:26.410Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.412Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5ecbfb30e120439b199cb644: start","job_status":"start","queue_duration_s":0.431702,"scheduling_latency_s":0.431728} gitlab # [ 218.213836] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.883Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5ecbfb30e120439b199cb644","created_at":"2026-07-09T06:52:26.410Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.412Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5ecbfb30e120439b199cb644: done: 0.039169 sec","job_status":"done","queue_duration_s":0.431702,"scheduling_latency_s":0.431728,"redis_calls":7,"redis_duration_s":0.003189,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039169,"completed_at":"2026-07-09T06:52:26.883Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00941,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 218.222124] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.885Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4d0470cc38fd7f61f7041a00","created_at":"2026-07-09T06:52:26.526Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.529Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4d0470cc38fd7f61f7041a00: start","job_status":"start","queue_duration_s":0.356523,"scheduling_latency_s":0.356548} gitlab # [ 218.296056] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.910Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4d0470cc38fd7f61f7041a00","created_at":"2026-07-09T06:52:26.526Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.529Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4d0470cc38fd7f61f7041a00: done: 0.024286 sec","job_status":"done","queue_duration_s":0.356523,"scheduling_latency_s":0.356548,"redis_calls":7,"redis_duration_s":0.002929,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000822,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024286,"completed_at":"2026-07-09T06:52:26.909Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.365276] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.912Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3a4f1c35693f2aeb3209be8d","created_at":"2026-07-09T06:52:26.537Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.539Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3a4f1c35693f2aeb3209be8d: start","job_status":"start","queue_duration_s":0.372604,"scheduling_latency_s":0.372628} gitlab # [ 218.366877] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.017Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3a4f1c35693f2aeb3209be8d","created_at":"2026-07-09T06:52:26.537Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.539Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3a4f1c35693f2aeb3209be8d: done: 0.105661 sec","job_status":"done","queue_duration_s":0.372604,"scheduling_latency_s":0.372628,"redis_calls":7,"redis_duration_s":0.00325,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.073,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.073,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.105661,"completed_at":"2026-07-09T06:52:27.017Z","load_balancing_strategy":"primary","db_duration_s":0.073006,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.434418] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.021Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"b090520daf566100bd301f34","created_at":"2026-07-09T06:52:26.549Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.550Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b090520daf566100bd301f34: start","job_status":"start","queue_duration_s":0.470983,"scheduling_latency_s":0.471032} gitlab # [ 218.435887] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.051Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"b090520daf566100bd301f34","created_at":"2026-07-09T06:52:26.549Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.550Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b090520daf566100bd301f34: done: 0.029552 sec","job_status":"done","queue_duration_s":0.470983,"scheduling_latency_s":0.471032,"redis_calls":10,"redis_duration_s":0.002908,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000768,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000731,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029552,"completed_at":"2026-07-09T06:52:27.051Z","load_balancing_strategy":"primary","db_duration_s":0.002964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.512530] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.053Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e487af4b9a28bf2467e574be","created_at":"2026-07-09T06:52:26.551Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.552Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e487af4b9a28bf2467e574be: start","job_status":"start","queue_duration_s":0.501017,"scheduling_latency_s":0.50106} gitlab # [ 218.519445] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e487af4b9a28bf2467e574be","created_at":"2026-07-09T06:52:26.551Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.552Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e487af4b9a28bf2467e574be: done: 0.023218 sec","job_status":"done","queue_duration_s":0.501017,"scheduling_latency_s":0.50106,"redis_calls":7,"redis_duration_s":0.0017759999999999998,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001321,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023218,"completed_at":"2026-07-09T06:52:27.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00301,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.593398] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.078Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"53f31fb29d42ea55d96ce935","created_at":"2026-07-09T06:52:26.552Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.554Z","job_size_bytes":19,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-53f31fb29d42ea55d96ce935: start","job_status":"start","queue_duration_s":0.523136,"scheduling_latency_s":0.52316} gitlab # [ 218.612316] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.098Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"53f31fb29d42ea55d96ce935","created_at":"2026-07-09T06:52:26.552Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.554Z","job_size_bytes":19,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-53f31fb29d42ea55d96ce935: done: 0.020881 sec","job_status":"done","queue_duration_s":0.523136,"scheduling_latency_s":0.52316,"redis_calls":8,"redis_duration_s":0.001892,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001464,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020881,"completed_at":"2026-07-09T06:52:27.098Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.682652] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.100Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e8404433e4d5887ca8e8fd93","created_at":"2026-07-09T06:52:26.555Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.556Z","job_size_bytes":66,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e8404433e4d5887ca8e8fd93: start","job_status":"start","queue_duration_s":0.543354,"scheduling_latency_s":0.543378} gitlab # [ 218.706269] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.116Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e8404433e4d5887ca8e8fd93","created_at":"2026-07-09T06:52:26.555Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.556Z","job_size_bytes":66,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e8404433e4d5887ca8e8fd93: done: 0.016227 sec","job_status":"done","queue_duration_s":0.543354,"scheduling_latency_s":0.543378,"redis_calls":7,"redis_duration_s":0.001702,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016227,"completed_at":"2026-07-09T06:52:27.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.786548] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.117Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"53e3b805aae109cb61fbd1d4","created_at":"2026-07-09T06:52:26.557Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.559Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-53e3b805aae109cb61fbd1d4: start","job_status":"start","queue_duration_s":0.558783,"scheduling_latency_s":0.558804} gitlab # [ 218.799196] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.146Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"53e3b805aae109cb61fbd1d4","created_at":"2026-07-09T06:52:26.557Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.559Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-53e3b805aae109cb61fbd1d4: done: 0.028402 sec","job_status":"done","queue_duration_s":0.558783,"scheduling_latency_s":0.558804,"redis_calls":7,"redis_duration_s":0.0018549999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028402,"completed_at":"2026-07-09T06:52:27.146Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003886,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.907935] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.148Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"ded8e024f076b2166788b6c6","created_at":"2026-07-09T06:52:26.595Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.597Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ded8e024f076b2166788b6c6: start","job_status":"start","queue_duration_s":0.550593,"scheduling_latency_s":0.550631} gitlab # [ 218.909679] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.170Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"ded8e024f076b2166788b6c6","created_at":"2026-07-09T06:52:26.595Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.597Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ded8e024f076b2166788b6c6: done: 0.022224 sec","job_status":"done","queue_duration_s":0.550593,"scheduling_latency_s":0.550631,"redis_calls":7,"redis_duration_s":0.00179,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022224,"completed_at":"2026-07-09T06:52:27.170Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.011297] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.171Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"9f3a73a5de0ed344d06a1eb8","created_at":"2026-07-09T06:52:26.598Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.599Z","job_size_bytes":6,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-9f3a73a5de0ed344d06a1eb8: start","job_status":"start","queue_duration_s":0.572469,"scheduling_latency_s":0.572493} gitlab # [ 219.012981] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"9f3a73a5de0ed344d06a1eb8","created_at":"2026-07-09T06:52:26.598Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.599Z","job_size_bytes":6,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-9f3a73a5de0ed344d06a1eb8: done: 0.013708 sec","job_status":"done","queue_duration_s":0.572469,"scheduling_latency_s":0.572493,"redis_calls":7,"redis_duration_s":0.001596,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013708,"completed_at":"2026-07-09T06:52:27.185Z","load_balancing_strategy":"primary","db_duration_s":0.000426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.112660] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.186Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"84b9333438f37c11b04887a0","created_at":"2026-07-09T06:52:26.599Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.603Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-84b9333438f37c11b04887a0: start","job_status":"start","queue_duration_s":0.582969,"scheduling_latency_s":0.582991} gitlab # [ 219.114504] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.216Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"84b9333438f37c11b04887a0","created_at":"2026-07-09T06:52:26.599Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.603Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-84b9333438f37c11b04887a0: done: 0.029812 sec","job_status":"done","queue_duration_s":0.582969,"scheduling_latency_s":0.582991,"redis_calls":10,"redis_duration_s":0.002517,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000621,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000391391,"duration_s":0.029812,"completed_at":"2026-07-09T06:52:27.216Z","load_balancing_strategy":"primary","db_duration_s":0.004173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.142601] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.218Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"75801ddd344521dcf0c3828e","created_at":"2026-07-09T06:52:26.835Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.836Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-75801ddd344521dcf0c3828e: start","job_status":"start","queue_duration_s":0.381147,"scheduling_latency_s":0.381167} gitlab # [ 219.216507] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.359Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"75801ddd344521dcf0c3828e","created_at":"2026-07-09T06:52:26.835Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.836Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-75801ddd344521dcf0c3828e: done: 0.141103 sec","job_status":"done","queue_duration_s":0.381147,"scheduling_latency_s":0.381167,"gitaly_calls":3,"gitaly_duration_s":0.010853,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011912,"redis_read_bytes":98,"redis_write_bytes":6011,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001692,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003198,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000315,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1130,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002494,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000255,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00089,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091598,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.116770427,"duration_s":0.141103,"completed_at":"2026-07-09T06:52:27.359Z","load_balancing_strategy":"primary","db_duration_s":0.03544,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 219.248297] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.361Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9a190ff218e5aef8eaf4677a","created_at":"2026-07-09T06:52:27.272Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-011c036e35534b1a","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:27.274Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9a190ff218e5aef8eaf4677a: start","job_status":"start","queue_duration_s":0.086835,"scheduling_latency_s":0.086868} gitlab # [ 219.316803] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.402Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9a190ff218e5aef8eaf4677a","created_at":"2026-07-09T06:52:27.272Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-011c036e35534b1a","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:27.274Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9a190ff218e5aef8eaf4677a: done: 0.041261 sec","job_status":"done","queue_duration_s":0.086835,"scheduling_latency_s":0.086868,"redis_calls":7,"redis_duration_s":0.002725,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001941,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041261,"completed_at":"2026-07-09T06:52:27.402Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004476,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.340792] gitaly[1397]: time="2026-07-09T06:52:27.286Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:57.284" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:27.285" grpc.time_ms=1.144 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.419229] gitaly[1397]: time="2026-07-09T06:52:27.315Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:57.314" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:27.314" grpc.time_ms=0.578 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.420598] gitaly[1397]: time="2026-07-09T06:52:27.345Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:57.342" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:27.343" grpc.time_ms=1.049 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.532801] gitaly[1397]: time="2026-07-09T06:52:28.336Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCD86EQ7S95092DK22R62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:58.233" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:28.334" grpc.time_ms=1.244 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.579114] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 85 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 1.53 seconds) gitlab # [ 219.668997] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:28.471Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2e28f00dfbd44ec845d9f76f","created_at":"2026-07-09T06:52:03.279Z","trace_propagation_headers":{"sentry-trace":"061fb86fd6484e9ba28c8e8040eccec1-dcd0ae1de21e48f2","baggage":"sentry-trace_id=061fb86fd6484e9ba28c8e8040eccec1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMDPE961CPQ6NT2P4CPQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:f204d91a34b669040ce6ef9ba5775f36f95b247b33712ac224f36df15364d890","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:28.467Z","retry_count":0,"job_size_bytes":451,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2e28f00dfbd44ec845d9f76f: start","job_status":"start","queue_duration_s":0.003517,"scheduling_latency_s":0.003588} gitlab # [ 219.762197] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:28.564Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2e28f00dfbd44ec845d9f76f","created_at":"2026-07-09T06:52:03.279Z","trace_propagation_headers":{"sentry-trace":"061fb86fd6484e9ba28c8e8040eccec1-dcd0ae1de21e48f2","baggage":"sentry-trace_id=061fb86fd6484e9ba28c8e8040eccec1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMDPE961CPQ6NT2P4CPQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:f204d91a34b669040ce6ef9ba5775f36f95b247b33712ac224f36df15364d890","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:28.467Z","retry_count":0,"job_size_bytes":451,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2e28f00dfbd44ec845d9f76f: fail: 0.093161 sec","job_status":"fail","queue_duration_s":0.003517,"scheduling_latency_s":0.003588,"redis_calls":7,"redis_duration_s":0.0032170000000000002,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.093161,"completed_at":"2026-07-09T06:52:28.564Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.006857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses gitlab # [ 220.730540] gitaly[1397]: time="2026-07-09T06:52:29.532Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCECFBRD9WH63TCBHRM25 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:59.430" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:29.530" grpc.time_ms=1.422 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.797668] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 143 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.21 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses gitlab # [ 221.813803] nix-daemon[3445]: accepted connection from pid 3685, user nixbot gitlab # [ 221.898437] gitaly[1397]: time="2026-07-09T06:52:30.700Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCFHVYV47RECF3NMMSAG5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:53:00.598" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:30.699" grpc.time_ms=0.712 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.936334] gitaly[1397]: time="2026-07-09T06:52:30.739Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCFHVYV47RECF3NMMSAG5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:40.257" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:30.727" grpc.time_ms=11.7 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.979485] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:30.782Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"54810a033250b68d627c0fbc","created_at":"2026-07-09T06:52:30.777Z","trace_propagation_headers":{"sentry-trace":"a282f290df694797b2b64649dcf2747d-8a915df8d107472b","baggage":"sentry-trace_id=a282f290df694797b2b64649dcf2747d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TCFHVYV47RECF3NMMSAG5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:30.780Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m7sj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-54810a033250b68d627c0fbc: start","job_status":"start","queue_duration_s":0.001516,"scheduling_latency_s":0.001553} gitlab # [ 221.994315] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 145 gitlab # [ 221.997553] gitaly[1397]: time="2026-07-09T06:52:30.800Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCFJ35AQ1Y357RJ8GFVTS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:53:00.699" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:30.799" grpc.time_ms=0.903 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.020704] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 164 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.22 seconds) (finished: subtest: eval and build post commit statuses, in 10.14 seconds) (finished: run the VM test script, in 258.02 seconds) gitlab # [ 222.032603] nixbot[3109]: {"ts": "2026-07-09T06:52:30.835Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf \"HTTP/1.1 201 Created\""} test script finished in 258.06s cleanup kill QemuMachine (pid 13) gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13) (finished: cleanup, in 0.02 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-09T06:52:16.803Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-09T06:52:20.675959Z', 'started_at': None, 'finished_at': None}] [] [{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': '2026-07-09T06:52:26.242Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-09T06:52:26.638Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': '2026-07-09T06:52:26.242Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-09T06:52:26.638Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': '2026-07-09T06:52:26.242Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-09T06:52:26.638Z', 'started_at': None, 'finished_at': '2026-07-09T06:52:30.768Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]