tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 13) gitlab # Formatting '/build/vm-state-gitlab/tmp.UQK9hCaqLh', fmt=raw size=1073741824 gitlab # mke2fs 1.47.3 (8-Jul-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: d636ea67-42bb-41b3-acc7-4bad81ac0809 (finished: start all VMs, in 38.68 seconds) gitlab # Superblock backups stored on blocks: gitlab: waiting for unit setup-gitlab.service gitlab # 32768, 98304, 163840, 229376 gitlab: waiting for the VM to finish booting gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/s3h9xdz84vf5m06gmzsr7crhj0z2sym0-nixos-system-gitlab-test/init regInfo=/nix/store/djfk9h995y1pzcnyy8444rhqa3bmz36h-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 560081440 cycles gitlab # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000008] tsc: Detected 2400.010 MHz processor gitlab # [ 0.000828] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000865] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000868] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000921] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002780] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002791] Using GB pages for direct mapping gitlab # [ 0.002974] RAMDISK: [mem 0xbe4e0000-0xbffcffff] gitlab # [ 0.002982] ACPI: Early table checksum verification disabled gitlab # [ 0.002985] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002988] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002992] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002999] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003001] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.003002] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003004] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003006] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003007] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.003008] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.003008] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.003009] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.003009] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.003010] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003498] No NUMA configuration found gitlab # [ 0.003500] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003503] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.005871] Zone ranges: gitlab # [ 0.005872] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005873] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005874] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005875] Device empty gitlab # [ 0.005876] Movable zone start for each node gitlab # [ 0.005877] Early memory node ranges gitlab # [ 0.005877] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005878] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005879] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005880] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005900] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.006438] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.118158] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.119181] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.119198] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.119236] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.119239] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.119240] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.119241] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.119242] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.119243] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.119246] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.119247] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.119253] TSC deadline timer available gitlab # [ 0.119258] CPU topo: Max. logical packages: 1 gitlab # [ 0.119259] CPU topo: Max. logical dies: 1 gitlab # [ 0.119259] CPU topo: Max. dies per package: 1 gitlab # [ 0.119262] CPU topo: Max. threads per core: 1 gitlab # [ 0.119263] CPU topo: Num. cores per package: 4 gitlab # [ 0.119263] CPU topo: Num. threads per package: 4 gitlab # [ 0.119263] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.119287] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.119305] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.119308] kvm-guest: setup PV sched yield gitlab # [ 0.119336] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.119338] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.119339] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.119340] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.119342] Booting paravirtualized kernel on KVM gitlab # [ 0.119345] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.123839] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.126153] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.126204] kvm-guest: PV spinlocks enabled gitlab # [ 0.126206] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.126209] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/s3h9xdz84vf5m06gmzsr7crhj0z2sym0-nixos-system-gitlab-test/init regInfo=/nix/store/djfk9h995y1pzcnyy8444rhqa3bmz36h-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.126302] Unknown kernel command line parameters "regInfo=/nix/store/djfk9h995y1pzcnyy8444rhqa3bmz36h-closure-info/registration", will be passed to user space. gitlab # [ 0.126322] random: crng init done gitlab # [ 0.126323] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.135477] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.139939] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.140018] software IO TLB: area num 4. gitlab # [ 0.216887] Fallback order for Node 0: 0 gitlab # [ 0.216896] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.216899] Policy zone: Normal gitlab # [ 0.219516] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.229138] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.243381] allocated 12582912 bytes of page_ext gitlab # [ 0.253342] ftrace: allocating 48565 entries in 192 pages gitlab # [ 0.253344] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.254252] Dynamic Preempt: lazy gitlab # [ 0.254452] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.254453] rcu: RCU event tracing is enabled. gitlab # [ 0.254454] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.254455] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.254456] Rude variant of Tasks RCU enabled. gitlab # [ 0.254456] Tracing variant of Tasks RCU enabled. gitlab # [ 0.254457] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.254457] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.254480] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.254482] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.254483] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.258859] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.259198] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.259321] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.266574] Console: colour VGA+ 80x25 gitlab # [ 0.266578] printk: legacy console [tty0] enabled gitlab # [ 0.310338] printk: legacy console [ttyS0] enabled gitlab # [ 0.506527] ACPI: Core revision 20250807 gitlab # [ 0.508154] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.510975] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.512701] x2apic enabled gitlab # [ 0.513917] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.515634] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.517833] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.520505] kvm-guest: setup PV IPIs gitlab # [ 0.522866] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.524663] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 0.527674] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010) gitlab # [ 0.529788] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.530827] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.531673] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.532678] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.535673] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.536674] Transient Scheduler Attacks: Mitigation: Clear CPU buffers gitlab # [ 0.538673] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.540673] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.541673] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.542679] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.544673] active return thunk: srso_alias_return_thunk gitlab # [ 0.545700] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.547673] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.548673] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.550673] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.552673] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.553673] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.554673] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.555673] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers' gitlab # [ 0.557673] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)' gitlab # [ 0.559674] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.560673] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.561673] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.563673] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.565673] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.566682] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16 gitlab # [ 0.567673] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24 gitlab # [ 0.568673] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format. gitlab # [ 0.598518] Freeing SMP alternatives memory: 44K gitlab # [ 0.599675] pid_max: default: 32768 minimum: 301 gitlab # [ 0.600789] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.602759] landlock: Up and running. gitlab # [ 0.603673] Yama: becoming mindful. gitlab # [ 0.605790] LSM support for eBPF active gitlab # [ 0.606932] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.607825] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.611120] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.612339] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.613683] ... version: 2 gitlab # [ 0.614675] ... bit width: 48 gitlab # [ 0.615675] ... generic counters: 6 gitlab # [ 0.616675] ... generic bitmap: 000000000000003f gitlab # [ 0.617675] ... fixed-purpose counters: 0 gitlab # [ 0.618675] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.619675] ... value mask: 0000ffffffffffff gitlab # [ 0.620675] ... max period: 00007fffffffffff gitlab # [ 0.621675] ... global_ctrl mask: 000000000000003f gitlab # [ 0.622796] signal: max sigframe size: 3376 gitlab # [ 0.623762] rcu: Hierarchical SRCU implementation. gitlab # [ 0.624680] rcu: Max phase no-delay instances is 400. gitlab # [ 0.625867] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.631227] smp: Bringing up secondary CPUs ... gitlab # [ 0.632092] smpboot: x86: Booting SMP configuration: gitlab # [ 0.632679] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.636687] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.638676] smpboot: Total of 4 processors activated (19200.08 BogoMIPS) gitlab # [ 0.640290] Memory: 6013364K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258300K reserved, 0K cma-reserved) gitlab # [ 0.642675] devtmpfs: initialized gitlab # [ 0.643946] x86/mm: Memory block size: 128MB gitlab # [ 0.646849] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.648727] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.649823] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.650788] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.652029] PM: RTC time: 10:08:50, date: 2026-07-06 gitlab # [ 0.655864] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.658027] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.658736] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.660814] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.661688] audit: initializing netlink subsys (disabled) gitlab # [ 0.662843] audit: type=2000 audit(1783332530.195:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.662989] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.664675] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.665676] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.666675] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.667676] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.668716] cpuidle: using governor menu gitlab # [ 0.671898] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.673009] PCI: Using configuration type 1 for base access gitlab # [ 0.673676] PCI: Using configuration type 1 for extended access gitlab # [ 0.674807] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.677024] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.678675] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.679676] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.680675] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.685039] ACPI: Added _OSI(Module Device) gitlab # [ 0.688686] ACPI: Added _OSI(Processor Device) gitlab # [ 0.692689] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.701691] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.711567] ACPI: Interpreter enabled gitlab # [ 0.715005] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.718686] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.722139] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.728683] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.734443] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.750078] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.753691] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.759927] acpiphp: Slot [3] registered gitlab # [ 0.762811] acpiphp: Slot [4] registered gitlab # [ 0.764792] acpiphp: Slot [5] registered gitlab # [ 0.767833] acpiphp: Slot [6] registered gitlab # [ 0.769788] acpiphp: Slot [7] registered gitlab # [ 0.772795] acpiphp: Slot [8] registered gitlab # [ 0.773737] acpiphp: Slot [9] registered gitlab # [ 0.775758] acpiphp: Slot [10] registered gitlab # [ 0.776747] acpiphp: Slot [11] registered gitlab # [ 0.777921] acpiphp: Slot [12] registered gitlab # [ 0.779745] acpiphp: Slot [13] registered gitlab # [ 0.781770] acpiphp: Slot [14] registered gitlab # [ 0.785824] acpiphp: Slot [15] registered gitlab # [ 0.788765] acpiphp: Slot [16] registered gitlab # [ 0.791764] acpiphp: Slot [17] registered gitlab # [ 0.794765] acpiphp: Slot [18] registered gitlab # [ 0.797809] acpiphp: Slot [19] registered gitlab # [ 0.800765] acpiphp: Slot [20] registered gitlab # [ 0.803786] acpiphp: Slot [21] registered gitlab # [ 0.807766] acpiphp: Slot [22] registered gitlab # [ 0.810796] acpiphp: Slot [23] registered gitlab # [ 0.813765] acpiphp: Slot [24] registered gitlab # [ 0.816765] acpiphp: Slot [25] registered gitlab # [ 0.819799] acpiphp: Slot [26] registered gitlab # [ 0.822776] acpiphp: Slot [27] registered gitlab # [ 0.824748] acpiphp: Slot [28] registered gitlab # [ 0.827747] acpiphp: Slot [29] registered gitlab # [ 0.829772] acpiphp: Slot [30] registered gitlab # [ 0.832747] acpiphp: Slot [31] registered gitlab # [ 0.834809] PCI host bridge to bus 0000:00 gitlab # [ 0.837708] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.839677] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.841681] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.843714] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.849691] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.855690] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.860972] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.870000] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.879045] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.890876] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.894878] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.899684] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.903684] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.907692] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.912314] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.919070] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.922168] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.925357] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.927690] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.930111] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.939768] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.941731] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.943728] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.945902] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.959683] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 13671 usecs gitlab # [ 0.962811] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.968717] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.970690] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.971729] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.973690] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.978330] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.984714] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.985695] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.990860] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 1.003140] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 1.017711] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 1.021715] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 1.025795] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 1.034390] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 1.041715] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 1.042694] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 1.044737] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 1.050491] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.061720] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 1.064693] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 1.066728] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 1.070753] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.077719] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 1.079690] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 1.080728] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 1.085703] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 1.090705] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 1.092690] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 1.093730] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 1.096690] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 1.100736] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 1.104421] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 1.106728] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 1.110730] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 1.116733] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 1.120720] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 1.124815] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 1.135906] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 1.149718] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 1.152718] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 1.157846] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 1.177058] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 1.178890] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 1.181343] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 1.186175] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 1.190958] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 1.199159] iommu: Default domain type: Translated gitlab # [ 1.202682] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 1.208258] ACPI: bus type USB registered gitlab # [ 1.210877] usbcore: registered new interface driver usbfs gitlab # [ 1.215702] usbcore: registered new interface driver hub gitlab # [ 1.216704] usbcore: registered new device driver usb gitlab # [ 1.221765] NetLabel: Initializing gitlab # [ 1.224685] NetLabel: domain hash size = 128 gitlab # [ 1.228685] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 1.233916] NetLabel: unlabeled traffic allowed by default gitlab # [ 1.237814] PCI: Using ACPI for IRQ routing gitlab # [ 1.242801] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 1.243670] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 1.243670] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 1.256696] vgaarb: loaded gitlab # [ 1.259276] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 1.263682] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 1.273672] clocksource: Switched to clocksource kvm-clock gitlab # [ 1.280150] VFS: Disk quotas dquot_6.6.0 gitlab # [ 1.282170] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 1.288266] pnp: PnP ACPI init gitlab # [ 1.292454] pnp: PnP ACPI: found 6 devices gitlab # [ 1.307784] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 1.314171] clocksource: Switched to clocksource acpi_pm gitlab # [ 1.318687] NET: Registered PF_INET protocol family gitlab # [ 1.326055] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 1.352661] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 1.355205] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 1.357867] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 1.370259] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 1.375977] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 1.381981] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 1.388303] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.394101] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.399884] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 1.403342] NET: Registered PF_XDP protocol family gitlab # [ 1.406344] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.409968] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.413551] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.417399] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.419822] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.422301] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.426869] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.435769] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.444189] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x7c0 took 12444 usecs gitlab # [ 1.450267] PCI: CLS 0 bytes, default 64 gitlab # [ 1.453851] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.454322] Trying to unpack rootfs image as initramfs... gitlab # [ 1.454757] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB) gitlab # [ 1.469026] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 1.501861] Initialise system trusted keyrings gitlab # [ 1.503916] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.527813] Key type asymmetric registered gitlab # [ 1.529118] Asymmetric key parser 'x509' registered gitlab # [ 1.530721] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.533424] io scheduler mq-deadline registered gitlab # [ 1.534866] io scheduler kyber registered gitlab # [ 1.540697] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.542953] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.546993] Linux agpgart interface v0.103 gitlab # [ 1.548335] ACPI: bus type drm_connector registered gitlab # [ 1.552095] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.554036] usbserial: USB Serial support registered for generic gitlab # [ 1.555819] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.555819] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.559741] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.562085] drop_monitor: Initializing network drop monitor service gitlab # [ 1.564050] NET: Registered PF_INET6 protocol family gitlab # [ 1.567023] Segment Routing with IPv6 gitlab # [ 1.568379] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.573790] IPI shorthand broadcast: enabled gitlab # [ 1.587381] sched_clock: Marking stable (1330095626, 256465092)->(1657143946, -70583228) gitlab # [ 1.594079] registered taskstats version 1 gitlab # [ 1.595697] Loading compiled-in X.509 certificates gitlab # [ 1.606291] Demotion targets for Node 0: null gitlab # [ 1.608141] Key type .fscrypt registered gitlab # [ 1.609381] Key type fscrypt-provisioning registered gitlab # [ 1.611049] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.612714] ima: Allocated hash algorithm: sha1 gitlab # [ 1.614140] ima: No architecture policies found gitlab # [ 1.615895] PM: Magic number: 2:580:122 gitlab # [ 1.617194] acpi LNXSYBUS:00: hash matches gitlab # [ 1.619377] RAS: Correctable Errors collector initialized. gitlab # [ 1.624725] clk: Disabling unused clocks gitlab # [ 1.625987] PM: genpd: Disabling unused power domains gitlab # [ 1.765680] Freeing initrd memory: 27584K gitlab # [ 1.769968] Freeing unused decrypted memory: 2028K gitlab # [ 1.775913] Freeing unused kernel image (initmem) memory: 3644K gitlab # [ 1.781040] Write protecting the kernel read-only data: 32768k gitlab # [ 1.788859] Freeing unused kernel image (text/rodata gap) memory: 1332K gitlab # [ 1.795634] Freeing unused kernel image (rodata/data gap) memory: 816K gitlab # [ 1.860958] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.863081] Run /init as init process gitlab # [ 1.878950] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.907937] fuse: init (API version 7.45) gitlab # [ 1.929716] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.939905] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.946106] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.996859] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 2.336962] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 2.358111] systemd[1]: Detected virtualization kvm. gitlab # [ 2.361991] systemd[1]: Detected architecture x86-64. gitlab # [ 2.365916] systemd[1]: Running in initrd. gitlab # [ 2.370197] systemd[1]: Initializing machine ID from random generator. gitlab # [ 2.375290] systemd[1]: Hostname set to . gitlab # [ 2.457784] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.622045] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.624124] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.628753] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.633864] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.638783] systemd[1]: Reached target Path Units. gitlab # [ 2.642772] systemd[1]: Reached target Slice Units. gitlab # [ 2.646384] systemd[1]: Reached target Swaps. gitlab # [ 2.649281] systemd[1]: Reached target Timer Units. gitlab # [ 2.652796] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.656945] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.660963] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.663808] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.665652] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.667327] systemd[1]: Reached target Socket Units. gitlab # [ 2.673412] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.679535] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.691126] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.696140] systemd[1]: Starting Journal Service... gitlab # [ 2.702427] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.704128] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 2.719929] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.734277] systemd-journald[152]: Collecting audit messages is disabled. gitlab # [ 2.741629] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.745896] netfs: FS-Cache loaded gitlab # [ 2.746851] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.754971] 9pnet: Installing 9P2000 support gitlab # [ 2.759245] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.765766] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.769896] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.770031] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.776705] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.778981] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.785061] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.789320] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.794218] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.805250] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.550511] systemd-modules-load[154]: Using 4 probe threads gitlab # [ 2.553084] systemd-modules-load[154]: Inserted module 'virtio_balloon' gitlab # [ 2.809314] systemd[1]: Started Journal Service. gitlab # [ 2.554839] systemd-modules-load[154]: Inserted module 'virtio_gpu' gitlab # [ 2.557103] systemd-modules-load[154]: Inserted module 'dm_mod' gitlab # [ 2.560645] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.567181] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.588979] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.590666] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.592257] systemd[1]: Reached target Local File Systems. gitlab # [ 2.595114] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.598953] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.609427] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.620186] systemd-udevd[193]: Using default interface naming scheme 'v260'. gitlab # [ 2.636603] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.644458] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.646110] systemd[1]: Reached target System Initialization. gitlab # [ 2.647482] systemd[1]: Reached target Basic System. gitlab # [ 3.044241] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 3.048816] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 3.056679] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 3.058415] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 3.061073] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 3.063845] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 3.066683] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 3.069009] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 3.070816] usb usb1: Product: UHCI Host Controller gitlab # [ 3.072059] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd gitlab # [ 3.073638] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 3.075990] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 3.079024] SCSI subsystem initialized gitlab # [ 3.084083] hub 1-0:1.0: USB hub found gitlab # [ 3.085346] virtio_blk virtio5: [vdb] 11450712 512-byte logical blocks (5.86 GB/5.46 GiB) gitlab # [ 3.089290] hub 1-0:1.0: 2 ports detected gitlab # [ 3.106567] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 3.110449] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 3.112503] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.895547] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.902345] (udev-worker)[206[ 3.158656] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # ]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 3.164587] scsi host0: ata_piix gitlab # [ 2.911552] (udev-worker)[218]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.915411] (udev-worker)[218]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.918714] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 3.176629] scsi host1: ata_piix gitlab # [ 3.177676] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 3.179450] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.922492] systemd-vconsole-setup[227]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.928318] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.930314] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.931713] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.933551] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.951948] systemd-fsck[241]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.957704] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 3.308677] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 3.340342] ata2: found unknown device (class 0) gitlab # [ 3.345212] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 3.352664] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 3.417815] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 3.458887] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 3.478909] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 3.483661] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 3.487956] usb 1-1: Product: QEMU USB Tablet gitlab # [ 3.490715] usb 1-1: Manufacturer: QEMU gitlab # [ 3.493192] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 3.536936] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 3.549256] usbcore: registered new interface driver usbhid gitlab # [ 3.552329] usbhid: USB HID core driver gitlab # [ 3.564146] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 3.569184] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 3.483639] systemd[1]: Mounting /sysroot... gitlab # [ 3.916773] EXT4-fs (vda): mounted filesystem d636ea67-42bb-41b3-acc7-4bad81ac0809 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.665620] systemd[1]: Mounted /sysroot. gitlab # [ 3.668140] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.673525] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.685790] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.692447] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.697195] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.700615] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.709747] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.969360] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.969499] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.717441] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.719267] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.720635] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.723507] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.726903] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.728195] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.729634] systemd-sysroot-fstab-check[272]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.732315] systemd[1]: Reload requested from client PID 272 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.734882] systemd[1]: Reloading... gitlab # [ 3.944767] systemd[1]: Reloading finished in 211 ms. gitlab # [ 3.999817] systemd-sysroot-fstab-check[272]: Requesting initrd-fs.target/start/replace... gitlab # [ 4.136637] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.141570] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.148692] systemd-sysroot-fstab-check[272]: Requesting swap.target/start/replace... gitlab # [ 4.157823] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.160759] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 4.164629] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 4.167697] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 4.182807] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.191205] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.483938] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.543510] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.547173] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.554633] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.561964] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.593939] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.598539] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.616529] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.619314] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.626521] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.663811] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.667325] systemd[1]: Stopped target Basic System. gitlab # [ 4.670470] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.673776] systemd[1]: Stopped target Path Units. gitlab # [ 4.676811] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.681260] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.685749] systemd[1]: Stopped target Slice Units. gitlab # [ 4.688868] systemd[1]: Stopped target Socket Units. gitlab # [ 4.691897] systemd[1]: Stopped target System Initialization. gitlab # [ 4.695366] systemd[1]: Stopped target Swaps. gitlab # [ 4.697876] systemd[1]: Stopped target Timer Units. gitlab # [ 4.700658] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.703475] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.706430] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.709815] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.712159] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.716888] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.719211] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.720978] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.722473] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.724218] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.725469] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.727334] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.729149] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.730900] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.733069] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.734480] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.736110] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.737780] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.741554] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.743354] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.745071] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.753563] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.757967] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.762215] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.765422] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.767178] systemd[1]: systemd-udevd.service: Consumed 1.022s CPU time over 2.163s wall clock time, 26M memory peak. gitlab # [ 4.774416] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.778626] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.785686] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.787389] systemd[1]: Closed udev Control Socket. gitlab # [ 4.794819] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.796207] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.797948] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.800705] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.802628] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.807700] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.809973] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.816118] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.818802] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.822369] systemd[1]: Reached target Switch Root. gitlab # [ 4.825618] systemd[1]: Starting NixOS Activation... gitlab # [ 4.939348] initrd-nixos-activation-start[570]: booting system configuration /nix/store/s3h9xdz84vf5m06gmzsr7crhj0z2sym0-nixos-system-gitlab-test gitlab # [ 4.991746] initrd-nixos-activation-start[570]: running activation script... gitlab # [ 5.241197] initrd-nixos-activation-start[593]: setting up /etc... gitlab # [ 5.364207] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 5.368941] systemd[1]: Finished NixOS Activation. gitlab # [ 5.381617] systemd[1]: Starting Switch Root... gitlab # [ 5.420748] systemd[1]: Switching root. gitlab # [ 6.077564] systemd-journald[152]: Received SIGTERM from PID 1 (systemd). gitlab # [ 6.199306] NET: Registered PF_VSOCK protocol family gitlab # [ 6.571026] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 6.586605] systemd[1]: Detected virtualization kvm. gitlab # [ 6.589581] systemd[1]: Detected architecture x86-64. gitlab # [ 6.592681] systemd[1]: Detected first boot. gitlab # [ 6.596591] systemd[1]: Initializing machine ID from random generator. gitlab # [ 6.774116] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 6.843835] systemd[1]: Applying preset policy. gitlab # [ 7.007935] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 7.136041] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 7.138260] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 7.141304] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 7.144642] systemd[1]: Created slice Slice /system/getty. gitlab # [ 7.146694] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 7.148885] systemd[1]: Created slice User and Session Slice. gitlab # [ 7.150652] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 7.152632] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 7.154531] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 7.155779] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 7.157082] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 7.158538] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 7.159829] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 7.161217] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 7.162741] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 7.164310] systemd[1]: Reached target Path Units. gitlab # [ 7.165626] systemd[1]: Reached target Remote File Systems. gitlab # [ 7.167013] systemd[1]: Reached target Slice Units. gitlab # [ 7.168249] systemd[1]: Reached target Swaps. gitlab # [ 7.170438] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 7.172566] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 7.175878] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 7.177942] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 7.179620] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 7.181405] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.183818] systemd[1]: Listening on udev Control Socket. gitlab # [ 7.187433] systemd[1]: Mounting Huge Pages File System... gitlab # [ 7.190392] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 7.193424] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 7.197623] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 7.206051] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 7.211914] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 7.237614] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 7.239129] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 7.241420] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.243977] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 7.247814] systemd[1]: Mounting FUSE Control File System... gitlab # [ 7.249235] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.268255] systemd[1]: Starting Journal Service... gitlab # [ 7.275205] systemd[1]: Starting Load Kernel Modules... gitlab # [ 7.284850] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 7.292066] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 7.297048] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.306726] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 7.309896] systemd-journald[787]: Collecting audit messages is enabled. gitlab # [ 7.316287] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 7.317393] loop: module loaded gitlab # [ 7.317982] systemd[1]: Mounted Huge Pages File System. gitlab # [ 7.320616] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 7.322323] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 7.323895] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 7.325567] EXT4-fs (vda): re-mounted d636ea67-42bb-41b3-acc7-4bad81ac0809. gitlab # [ 7.326290] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 7.330224] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 7.333151] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 7.079509] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 7.335876] systemd[1]: Mounted FUSE Control File System. gitlab # [ 7.081740] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 7.340161] systemd[1]: Started Journal Service. gitlab # [ 7.084261] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 7.087193] systemd-modules-load[788]: Using 3 probe threads gitlab # [ 7.088961] systemd-modules-load[788]: Inserted module 'tls' gitlab # [ 7.090692] systemd-modules-load[788]: Inserted module 'loop' gitlab # [ 7.092638] systemd-oomd[789]: No swap; memory pressure usage will be degraded gitlab # [ 7.095257] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 7.097133] systemd[1]: Finished Load Kernel Modules. gitlab # [ 7.098693] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 7.108872] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 7.112533] systemd[1]: Starting Firewall... gitlab # [ 7.121307] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 7.123137] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.126193] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 7.127856] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 7.388786] systemd-journald[787]: Received client request to flush runtime journal. gitlab # [ 7.204047] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 7.206252] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.209510] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 7.211285] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 7.212837] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 7.214655] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 7.216597] systemd[1]: Reached target First Boot Complete. gitlab # [ 7.218137] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 7.221520] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 7.224263] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 7.226583] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 7.228659] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 7.235637] systemd-udevd[825]: Using default interface naming scheme 'v260'. gitlab # [ 7.259108] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 7.273675] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 7.397054] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 7.412342] systemd[1]: Finished Firewall. gitlab # [ 7.426418] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 7.440615] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 7.442699] (udev-worker)[924]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.445850] (udev-worker)[926]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 7.449949] (udev-worker)[926]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.453846] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 7.455757] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.458314] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.461952] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.464662] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.467192] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.751946] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 7.776769] ACPI: button: Power Button [PWRF] gitlab # [ 7.790612] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 7.792099] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 7.794950] mousedev: PS/2 mouse device common for all mice gitlab # [ 7.804780] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 7.816576] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 7.819169] FDC 0 is a S82078B gitlab # [ 7.819893] rtc_cmos 00:05: registered as rtc0 gitlab # [ 7.822123] rtc_cmos 00:05: setting system clock to 2026-07-06T10:08:57 UTC (1783332537) gitlab # [ 7.825307] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 7.850110] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 7.851589] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 7.854677] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 7.867168] Console: switching to colour dummy device 80x25 gitlab # [ 7.869931] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 7.870808] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 7.872417] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 7.873786] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 7.876196] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 7.884719] fbcon: bochs-drmdrmfb (fb0) is primary device gitlab # [ 7.885596] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 7.890944] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 7.686402] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.921252] Console: switching to colour frame buffer device 160x50 gitlab # [ 7.948622] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 7.741142] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.743242] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.751105] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.013844] ppdev: user-space parallel port driver gitlab # [ 7.762481] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.764585] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.772102] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.047175] kvm_amd: TSC scaling supported gitlab # [ 8.048103] kvm_amd: Nested Virtualization enabled gitlab # [ 8.049047] kvm_amd: Nested Paging enabled gitlab # [ 8.049884] kvm_amd: LBR virtualization supported gitlab # [ 8.050835] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 8.051933] kvm_amd: Virtual GIF supported gitlab # [ 8.052871] kvm_amd: Virtual NMI enabled gitlab # [ 8.101658] EDAC MC: Ver: 3.0.0 gitlab # [ 7.854670] systemd-vconsole-setup[960]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 7.858913] systemd[1]: Finished Virtual Console Setup. gitlab # [ 7.881283] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.898225] systemd[1]: Mounted /run/wrappers. gitlab # [ 7.899481] systemd[1]: Reached target Local File Systems. gitlab # [ 7.900834] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 7.902478] systemd[1]: Starting register-nix-paths.service... gitlab # [ 7.903902] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 7.905380] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.908265] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 7.912660] systemd[1]: Starting Create System Files and Directories... gitlab # [ 7.938091] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 7.940249] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.945439] systemd[1]: Finished Create System Files and Directories. gitlab # [ 7.949631] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 7.951442] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 7.969485] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.985531] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.987708] systemd[1]: Starting Update is Completed... gitlab # [ 8.006693] systemd[1]: Finished Update is Completed. gitlab # [ 8.043043] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 8.044975] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 8.046480] systemd[1]: suid-sgid-wrappers.service: Consumed 114ms CPU time over 140ms wall clock time, 5.1M memory peak, 1.1M read from disk. gitlab # [ 8.299952] systemd[1]: Finished register-nix-paths.service. gitlab # [ 8.303912] systemd[1]: Reached target System Initialization. gitlab # [ 8.307947] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 8.312502] systemd[1]: Started logrotate.timer. gitlab # [ 8.315878] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 8.320203] systemd[1]: Reached target Timer Units. gitlab # [ 8.323573] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 8.327767] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 8.331472] systemd[1]: Starting nixbot CI service socket... gitlab # [ 8.335353] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 8.339262] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 8.343205] systemd[1]: Reached target Socket Units. gitlab # [ 8.346223] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 8.408932] dbus-broker-launch[1051]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 8.415394] dbus-broker-launch[1051]: NSS returned no entry for 'systemd-timesync' gitlab # [ 8.419839] dbus-broker-launch[1051]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 8.428824] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 8.451448] dbus-broker-launch[1051]: Ready gitlab # [ 8.457684] systemd[1]: Reached target Basic System. gitlab # [ 8.460907] systemd[1]: Started backdoor.service. gitlab # [ 8.463931] systemd[1]: Starting gitlab-config.service... gitlab # [ 8.467968] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 8.482412] systemd[1]: Starting Logrotate configuration check... gitlab # [ 8.490245] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.494336] systemd[1]: Starting Post-Boot Actions... gitlab # [ 8.501837] systemd[1]: Started Reset console on configuration changes. gitlab # [ 8.507544] logrotate[1056]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # connecting to host... gitlab # [ 8.510383] logrotate[1056]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf gitlab # [ 8.514716] logrotate[1056]: note: 'monthly' overrides previously specified 'weekly' gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab: connected to guest root shell gitlab # [ 8.516491] logrotate[1056]: Reading state from file: /var/lib/logrotate.status gitlab: (connecting took 9.41 seconds) gitlab # [ 8.518324] logrotate[1056]: state file /var/lib/logrotate.status does not exist gitlab: (finished: waiting for the VM to finish booting, in 9.41 seconds) gitlab # [ 8.519482] logrotate[1056]: Allocating hash table for state file, size 64 entries gitlab # [ 8.520628] logrotate[1056]: Handling 4 logs gitlab # [ 8.521384] logrotate[1056]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 8.522225] logrotate[1056]: considering log /var/log/btmp gitlab # [ 8.522978] logrotate[1056]: Creating new state gitlab # [ 8.524044] logrotate[1056]: Now: 2026-07-06 10:08 gitlab # [ 8.524786] logrotate[1056]: Last rotated at 2026-07-06 10:00 gitlab # [ 8.525824] logrotate[1056]: log does not need rotating (log has already been rotated) gitlab # [ 8.526872] logrotate[1056]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 8.527991] logrotate[1056]: considering log /var/log/wtmp gitlab # [ 8.528751] logrotate[1056]: Creating new state gitlab # [ 8.529770] logrotate[1056]: Now: 2026-07-06 10:08 gitlab # [ 8.530837] logrotate[1056]: Last rotated at 2026-07-06 10:00 gitlab # [ 8.531989] logrotate[1056]: log does not need rotating (log has already been rotated) gitlab # [ 8.533396] logrotate[1056]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 8.534805] logrotate[1056]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1056) gitlab # [ 8.535934] logrotate[1056]: considering log /var/gitlab/state/log/*.log gitlab # [ 8.537084] logrotate[1056]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 8.537880] logrotate[1056]: Creating new state gitlab # [ 8.538997] logrotate[1056]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1056) gitlab # [ 8.539912] logrotate[1056]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 8.541075] logrotate[1056]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1056) gitlab # [ 8.541874] logrotate[1056]: considering log /var/log/nginx/*.log gitlab # [ 8.542993] logrotate[1056]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 8.543842] logrotate[1056]: Creating new state gitlab # [ 8.544956] logrotate[1056]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1056) gitlab # [ 8.546259] systemd[1]: Starting resolvconf update... gitlab # [ 8.583405] systemd[1]: Finished Logrotate configuration check. gitlab # [ 8.585934] systemd[1]: Finished Post-Boot Actions. gitlab # [ 8.593521] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 8.605586] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.607780] nsncd[1057]: Jul 06 10:08:58.539 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.608982] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.610095] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.618114] systemd[1]: Starting User Login Management... gitlab # [ 8.644128] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 8.648386] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 8.649846] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 8.652235] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 8.654263] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 8.656164] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 8.659240] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 8.660358] systemd[1]: nscd.service: Consumed 15ms CPU time over 159ms wall clock time, 8M memory peak, 5.6M read from disk. gitlab # [ 8.661626] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.674601] systemd-logind[1111]: New seat seat0. gitlab # [ 8.675976] systemd[1]: Finished resolvconf update. gitlab # [ 8.678087] systemd[1]: Reached target Preparation for Network. gitlab # [ 8.681788] systemd-logind[1111]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 8.682965] systemd-logind[1111]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 8.684798] systemd-logind[1111]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 8.689216] systemd[1]: Starting DHCP Client... gitlab # [ 8.691464] systemd[1]: Starting Address configuration of eth1... gitlab # [ 8.693988] nsncd[1147]: Jul 06 10:08:58.627 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.699831] systemd[1]: Starting Extra networking commands.... gitlab # [ 8.702874] systemd[1]: Started User Login Management. gitlab # [ 8.703690] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.706604] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.707912] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.713059] systemd[1]: Starting linger-users.service... gitlab # [ 8.721888] network-addresses-eth1-start[1156]: adding address 192.168.1.1/24... done gitlab # [ 8.725088] network-addresses-eth1-start[1156]: adding address 2001:db8:1::1/64... done gitlab # [ 8.732921] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 8.733965] systemd[1]: Finished linger-users.service. gitlab # [ 8.735505] systemd[1]: Finished Address configuration of eth1. gitlab # [ 8.773092] systemd[1]: Finished Extra networking commands.. gitlab # [ 8.776195] systemd[1]: Reached target Network. gitlab # [ 8.777851] dhcpcd[1177]: dhcpcd-10.3.2 starting gitlab # [ 8.779651] dhcpcd[1227]: dev: loaded udev gitlab # [ 8.780893] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 8.783778] systemd[1]: Starting Nginx Web Server... gitlab # [ 8.785318] systemd[1]: Starting PostgreSQL Server... gitlab # [ 9.044138] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 9.045579] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.791327] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 8.794395] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.815538] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.819800] systemd[1]: Finished Permit User Sessions. gitlab # [ 8.825740] systemd[1]: Started Getty on tty1. gitlab # [ 8.827898] systemd[1]: Reached target Login Prompts. gitlab # [ 9.134207] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 9.140779] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 9.142103] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 9.144541] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 9.146301] cfg80211: failed to load regulatory.db gitlab # [ 9.188775] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 8.934659] dhcpcd[1227]: eth0: waiting for carrier gitlab # [ 8.936415] dhcpcd[1227]: eth0: carrier acquired gitlab # [ 8.940474] nginx-pre-start[1254]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok gitlab # [ 8.942585] nginx-pre-start[1254]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful gitlab # [ 8.943855] dhcpcd[1227]: DUID 00:01:00:01:31:de:3b:3a:52:54:00:12:34:56 gitlab # [ 8.944422] dhcpcd[1227]: eth0: IAID 00:12:34:56 gitlab # [ 8.945417] dhcpcd[1227]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 8.953891] systemd[1]: Started Nginx Web Server. gitlab # [ 8.961383] postgresql-pre-start[1258]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.964499] postgresql-pre-start[1258]: This user must also own the server process. gitlab # [ 8.966943] redis-server[1250]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 8.971087] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 8.972752] redis-server[1250]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 8.975134] postgresql-pre-start[1258]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.977710] postgresql-pre-start[1258]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.980255] postgresql-pre-start[1258]: The default text search configuration will be set to "english". gitlab # [ 8.982628] postgresql-pre-start[1258]: Data page checksums are disabled. gitlab # [ 8.984529] postgresql-pre-start[1258]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok gitlab # [ 8.986989] postgresql-pre-start[1258]: creating subdirectories ... ok gitlab # [ 8.989340] postgresql-pre-start[1258]: selecting dynamic shared memory implementation ... posix gitlab # [ 8.990381] redis-server[1250]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1250, just started gitlab # [ 8.991001] redis-server[1250]: Configuration loaded gitlab # [ 8.991822] redis[1250]: monotonic clock: POSIX clock_gettime gitlab # [ 8.999514] redis[1250]: Running mode=standalone, port=0. gitlab # [ 9.000993] redis[1250]: Server initialized gitlab # [ 9.003233] redis[1250]: Ready to accept connections unix gitlab # [ 9.004921] redis[1250]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 9.013219] postgresql-pre-start[1258]: selecting default "max_connections" ... 100 gitlab # [ 9.045937] postgresql-pre-start[1258]: selecting default "shared_buffers" ... 128MB gitlab # [ 9.285939] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 9.433095] postgresql-pre-start[1258]: selecting default time zone ... UTC gitlab # [ 9.435358] postgresql-pre-start[1258]: creating configuration files ... ok gitlab # [ 9.548282] gitlab-workhorse[1287]: time="2026-07-06T10:08:59Z" level=info msg=Starting build_time=19700101.000000 version=18.11.4 gitlab # [ 9.551161] gitlab-workhorse[1287]: time="2026-07-06T10:08:59Z" level=info msg="Using redis/go-redis" gitlab # [ 9.553536] gitlab-workhorse[1287]: time="2026-07-06T10:08:59Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 9.616164] postgresql-pre-start[1258]: running bootstrap script ... ok gitlab # [ 10.072344] postgresql-pre-start[1258]: performing post-bootstrap initialization ... ok gitlab # [ 10.270561] dhcpcd[1227]: eth0: soliciting a DHCP lease gitlab # [ 10.535913] NET: Registered PF_PACKET protocol family gitlab # [ 10.288525] dhcpcd[1227]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 10.297358] dhcpcd[1227]: eth0: probing address 10.0.2.15/24 gitlab # [ 10.413985] postgresql-pre-start[1258]: syncing data to disk ... ok gitlab # [ 10.416818] postgresql-pre-start[1258]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 10.420906] postgresql-pre-start[1258]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 10.424824] postgresql-pre-start[1258]: Success. You can now start the database server using: gitlab # [ 10.427799] postgresql-pre-start[1258]: pg_ctl -D /var/lib/postgresql/17 -l logfile start gitlab # [ 10.515952] postgres[1313]: [1313] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 10.523186] postgres[1313]: [1313] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 10.525924] postgres[1313]: [1313] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 10.535511] postgres[1313]: [1313] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 10.551719] postgres[1320]: [1320] LOG: database system was shut down at 2026-07-06 10:08:59 GMT gitlab # [ 10.564148] postgres[1313]: [1313] LOG: database system is ready to accept connections gitlab # [ 10.572179] systemd[1]: Started PostgreSQL Server. gitlab # [ 10.575502] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.747177] postgresql-setup-start[1335]: CREATE DATABASE gitlab # [ 10.789505] postgresql-setup-start[1345]: CREATE ROLE gitlab # [ 10.805730] postgresql-setup-start[1347]: ALTER DATABASE gitlab # [ 10.828488] postgresql-setup-start[1356]: CREATE ROLE gitlab # [ 10.833596] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 10.836213] systemd[1]: Reached target PostgreSQL. gitlab # [ 10.837450] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 10.950050] gitlab-postgresql-start[1365]: CREATE DATABASE gitlab # [ 10.961709] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1354]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 10.964819] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1354]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 11.013138] gitlab-postgresql-start[1375]: CREATE EXTENSION gitlab # [ 11.017935] systemd[1]: Finished gitlab-config.service. gitlab # [ 11.021453] systemd[1]: Started gitaly.service. gitlab # [ 11.024223] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 11.055080] gitlab-postgresql-start[1382]: CREATE EXTENSION gitlab # [ 11.062495] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 11.085215] dhcpcd[1227]: eth0: soliciting an IPv6 router gitlab # [ 11.088720] dhcpcd[1227]: eth0: Router Advertisement from fe80::2 gitlab # [ 11.090050] dhcpcd[1227]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 11.092967] dhcpcd[1227]: eth0: adding route to fec0::/64 gitlab # [ 11.096052] dhcpcd[1227]: eth0: adding default route via fe80::2 gitlab # [ 11.630714] gitaly[1385]: time="2026-07-06T10:09:01.563Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=1385 gitlab # [ 11.646603] gitaly[1385]: time="2026-07-06T10:09:01.564Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=1385 gitlab # [ 11.654833] gitaly[1385]: time="2026-07-06T10:09:01.564Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=1385 gitlab # [ 11.663753] gitaly[1385]: time="2026-07-06T10:09:01.564Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=1385 gitlab # [ 11.671581] gitaly[1385]: time="2026-07-06T10:09:01.564Z" level=info msg="Starting Gitaly" pid=1385 version=18.11.4 gitlab # [ 11.833128] gitaly[1385]: time="2026-07-06T10:09:01.767Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=201 pid=1385 gitlab # [ 11.836622] gitaly[1385]: time="2026-07-06T10:09:01.767Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=1385 gitlab # [ 11.839843] gitaly[1385]: time="2026-07-06T10:09:01.768Z" level=info msg="finished initializing command factory" duration_ms=1 pid=1385 gitlab # [ 11.843256] gitaly[1385]: time="2026-07-06T10:09:01.768Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1385/git-exec-3214504868.d/git pid=1385 gitlab # [ 11.847268] gitaly[1385]: time="2026-07-06T10:09:01.773Z" level=info msg="finished detecting git version" duration_ms=4 pid=1385 gitlab # [ 11.850445] gitaly[1385]: time="2026-07-06T10:09:01.773Z" level=info msg="using Git version" pid=1385 version=2.53.g gitlab # [ 11.853386] gitaly[1385]: time="2026-07-06T10:09:01.774Z" level=info msg="clearing disk cache object folder" pid=1385 storage=default gitlab # [ 11.856872] gitaly[1385]: time="2026-07-06T10:09:01.774Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3266470480 pid=1385 storage=default gitlab # [ 11.857883] gitaly[1385]: time="2026-07-06T10:09:01.774Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3266470480 pid=1385 storage=default gitlab # [ 11.859320] gitaly[1385]: time="2026-07-06T10:09:01.774Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=1385 gitlab # [ 11.861132] gitaly[1385]: time="2026-07-06T10:09:01.774Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=1385 gitlab # [ 11.862361] gitaly[1385]: time="2026-07-06T10:09:01.774Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3266470480 clear_duration_ms=0 pid=1385 storage=default gitlab # [ 11.911363] gitaly[1385]: time="2026-07-06T10:09:01.845Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=1385 gitlab # [ 11.913121] gitaly[1385]: time="2026-07-06T10:09:01.845Z" level=info msg="counting repositories" pid=1385 gitlab # [ 11.914140] gitaly[1385]: time="2026-07-06T10:09:01.845Z" level=info msg="finished tempdir cleaner walk" pid=1385 storage=default time_ms=0 gitlab # [ 11.915937] gitaly[1385]: time="2026-07-06T10:09:01.845Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1385 schema=unix gitlab # [ 11.917299] gitaly[1385]: time="2026-07-06T10:09:01.845Z" level=info msg="starting to count repositories" pid=1385 storage_path=/var/gitlab/state/repositories gitlab # [ 11.918492] gitaly[1385]: time="2026-07-06T10:09:01.845Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1385/sock.d/intern pid=1385 schema=unix gitlab # [ 11.919578] gitaly[1385]: time="2026-07-06T10:09:01.845Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1385 storage_path=/var/gitlab/state/repositories gitlab # [ 11.920981] gitaly[1385]: time="2026-07-06T10:09:01.845Z" level=info msg="completed counting all repositories" counting_duration=0 pid=1385 gitlab # [ 11.938561] gitaly[1385]: time="2026-07-06T10:09:01.872Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=1385 gitlab # [ 11.940423] gitaly[1385]: time="2026-07-06T10:09:01.874Z" level=info msg="Started Gitaly" duration_ms=309 pid=1385 gitlab # [ 11.941761] gitaly[1385]: time="2026-07-06T10:09:01.874Z" level=info msg="maintenance: daily scheduled" pid=1385 scheduled="2026-07-06 12:00:00 +0000 UTC" gitlab # [ 12.394665] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1388]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.398947] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1388]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.417577] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1388]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.846115] gitaly[1385]: time="2026-07-06T10:09:03.779Z" level=info msg="License database preloaded" duration_ms=2215 pid=1385 gitlab # [ 15.243597] dhcpcd[1227]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 15.245941] dhcpcd[1227]: eth0: adding route to 10.0.2.0/24 gitlab # [ 15.247676] dhcpcd[1227]: eth0: adding default route via 10.0.2.2 gitlab # [ 15.370430] systemd[1]: Started DHCP Client. gitlab # [ 15.374478] systemd[1]: Reached target Network is Online. gitlab # [ 42.584903] postgres[1736]: [1736] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 42.589300] postgres[1736]: [1736] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 42.591966] postgres[1736]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 42.595346] postgres[1736]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 42.600013] postgres[1736]: attidentity AS identity, gitlab # [ 42.604167] postgres[1736]: attgenerated as attgenerated gitlab # [ 42.608369] postgres[1736]: FROM pg_attribute a gitlab # [ 42.612438] postgres[1736]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 42.616921] postgres[1736]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 42.620842] postgres[1736]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 42.624674] postgres[1736]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 42.630306] postgres[1736]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 42.634234] postgres[1736]: ORDER BY a.attnum gitlab # [ 42.636574] postgres[1736]: gitlab # [ 42.786840] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1388]: W, [2026-07-06T10:09:32.720035 #1388] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 48.073402] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1800]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 48.076946] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1800]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 49.617622] gitlab-workhorse[1287]: time="2026-07-06T10:09:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 59.628901] gitlab-workhorse[1287]: time="2026-07-06T10:09:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 65.576968] postgres[1875]: [1875] ERROR: relation "table_without_models" does not exist gitlab # [ 65.581689] postgres[1875]: [1875] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 69.638738] gitlab-workhorse[1287]: time="2026-07-06T10:09:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.648089] gitlab-workhorse[1287]: time="2026-07-06T10:10:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 80.148106] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 80.154551] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 80.158810] systemd[1]: Starting gitlab.service... gitlab # [ 80.505949] bundle[2078]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 80.514707] bundle[2078]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 80.522828] bundle[2078]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 83.353967] sidekiq-cluster[2077]: {"severity":"INFO","time":"2026-07-06T10:10:13.270Z","message":"Starting cluster with 1 processes"} gitlab # [ 83.489258] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.401Z","pid":2078,"message":"* Enabling systemd notification integration"} gitlab # [ 83.495833] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.422Z","pid":2078,"message":"Puma starting in cluster mode..."} gitlab # [ 83.501937] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.422Z","pid":2078,"message":"* Puma version: 7.2.0 (\"On The Corner\")"} gitlab # [ 83.508527] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.422Z","pid":2078,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 83.516313] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.422Z","pid":2078,"message":"* Min threads: 0"} gitlab # [ 83.521817] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.422Z","pid":2078,"message":"* Max threads: 4"} gitlab # [ 83.527509] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.423Z","pid":2078,"message":"* Environment: production"} gitlab # [ 83.533363] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.423Z","pid":2078,"message":"* Master PID: 2078"} gitlab # [ 83.538815] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.423Z","pid":2078,"message":"* Workers: 2"} gitlab # [ 83.543098] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.423Z","pid":2078,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 83.548418] bundle[2078]: {"timestamp":"2026-07-06T10:10:13.423Z","pid":2078,"message":"* Preloading application"} gitlab # [ 89.659332] gitlab-workhorse[1287]: time="2026-07-06T10:10:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 99.665625] gitlab-workhorse[1287]: time="2026-07-06T10:10:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 107.284432] bundle[2078]: W, [2026-07-06T10:10:37.217678 #2078] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 109.675634] gitlab-workhorse[1287]: time="2026-07-06T10:10:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 110.067470] sidekiq-cluster[2108]: 2026-07-06T10:10:40.000Z pid=2108 tid=tys INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2108, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>428081152} gitlab # [ 110.254338] sidekiq-cluster[2108]: W, [2026-07-06T10:10:40.188155 #2108] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 110.298486] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:40.232Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 112.171984] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.105Z","message":"GitLab reliable fetch activated!"} gitlab # [ 112.175524] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.105Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 112.206278] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.140Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 112.212499] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.146Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 112.220550] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.154Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 112.224872] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.158Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 112.230152] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.164Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 112.235936] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.170Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 112.241443] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.175Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 112.247431] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.181Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 112.253258] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.187Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"} gitlab # [ 112.259251] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.193Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 112.265179] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.199Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 112.271643] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.205Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 112.277379] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.211Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 112.283232] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.217Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 112.290813] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.225Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 112.296835] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.231Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"} gitlab # [ 112.303194] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.237Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 112.310190] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.244Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 112.316291] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.250Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 112.323702] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.257Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 112.330710] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.264Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 112.337490] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.271Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 112.344098] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.278Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 112.350405] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.284Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 112.356797] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.291Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 112.363183] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.297Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 112.369516] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.303Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 112.375895] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.310Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 112.382339] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.316Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 112.388953] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.323Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 112.395776] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.329Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 112.402489] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.336Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 112.409700] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.343Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 112.416078] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.350Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 112.422709] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.356Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 112.428897] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.363Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 112.434751] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.368Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 112.440940] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.375Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 112.446774] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.380Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 112.452553] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.386Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 112.458242] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.392Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 112.464567] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.398Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 112.470744] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.404Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 112.477067] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.411Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 112.483251] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.417Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 112.489892] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.424Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 112.496233] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.430Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 112.503918] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.438Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 112.510166] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.444Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 112.516566] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.450Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 112.523229] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.457Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 112.529630] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.463Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 112.537247] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.471Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 112.550443] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.484Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 112.557230] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.491Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 112.563572] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.497Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 112.571772] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.505Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 112.578209] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.512Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 112.584964] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.519Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 112.591148] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.525Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 112.597558] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.531Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 112.604405] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.538Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 112.610231] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.544Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 112.617040] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.551Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 112.624367] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.558Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 112.630960] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.565Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 112.637874] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.572Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 112.643706] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.577Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 112.671347] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.605Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 112.677821] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.611Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 112.684222] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.618Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 112.690204] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.624Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 112.696112] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.630Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 112.703407] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.637Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 112.709904] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.644Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 112.716762] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.650Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 112.722870] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.657Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 112.729397] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.663Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 112.735915] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.670Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 112.742337] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.676Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 112.748226] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.682Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 112.752158] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.686Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 112.759349] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.693Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 112.765102] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.699Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 112.768700] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.701Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 112.770054] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.704Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 112.773049] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.707Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 112.776263] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.710Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 112.780108] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.712Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 112.780964] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.715Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 112.783943] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.718Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 112.787108] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.721Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 112.790664] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.724Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 112.793944] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.728Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 112.797787] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.732Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 112.801722] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.735Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 112.806063] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.740Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 112.808961] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.743Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 112.812523] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.746Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 112.823661] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.757Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 112.827569] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.761Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 112.831302] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.763Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 112.835652] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.769Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 112.839490] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.772Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 112.845966] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.780Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 112.850230] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.783Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 112.856426] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.790Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 112.860177] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.793Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 112.867849] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.802Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 112.874091] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:10:42.808Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 119.685441] gitlab-workhorse[1287]: time="2026-07-06T10:10:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 129.687927] gitlab-workhorse[1287]: time="2026-07-06T10:10:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 134.255962] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:04.189Z","message":"Booted Rails 7.2.3 application in production environment"} gitlab # [ 134.263161] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:04.197Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 134.271382] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:04.197Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 134.278627] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:04.197Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 134.286901] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:04.199Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 134.980151] bundle[2078]: {"timestamp":"2026-07-06T10:11:04.913Z","pid":2078,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 134.987640] bundle[2078]: {"timestamp":"2026-07-06T10:11:04.913Z","pid":2078,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 134.994616] bundle[2078]: {"timestamp":"2026-07-06T10:11:04.913Z","pid":2078,"message":"! #\u003cThread:0x00007d0e99449948 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 135.013949] bundle[2078]: {"timestamp":"2026-07-06T10:11:04.913Z","pid":2078,"message":"! #\u003cThread:0x00007d0e996737c8 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 135.030620] bundle[2078]: {"timestamp":"2026-07-06T10:11:04.913Z","pid":2078,"message":"Use Ctrl-C to stop"} gitlab # [ 135.034323] bundle[2078]: {"timestamp":"2026-07-06T10:11:04.914Z","pid":2078,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 135.381790] bundle[2078]: {"timestamp":"2026-07-06T10:11:05.315Z","pid":2078,"message":"- Worker 0 (PID: 2606) booted in 0.37s, phase: 0"} gitlab # [ 135.395787] bundle[2078]: {"timestamp":"2026-07-06T10:11:05.329Z","pid":2078,"message":"- Worker 1 (PID: 2610) booted in 0.36s, phase: 0"} gitlab # [ 135.404840] systemd[1]: Started gitlab.service. gitlab # [ 135.408214] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 135.412650] systemd[1]: Starting setup-gitlab.service... gitlab # [ 135.424299] setup-gitlab-start[2639]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 135.560120] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:05.493Z","message":"Cleaning working queues"} gitlab # [ 135.570729] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:05.504Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"892ea19e81c57dd66993d4ca","created_at":"2026-07-06T10:10:09.507Z","trace_propagation_headers":{"sentry-trace":"88bc0b434ba94974ae27f47ac44808c8-93793177d95547e9","baggage":"sentry-trace_id=88bc0b434ba94974ae27f47ac44808c8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"56430837907e8788211e7df331d90171","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T10:10:09.521Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-892ea19e81c57dd66993d4ca: start","job_status":"start","queue_duration_s":55.983157,"scheduling_latency_s":55.98326} gitlab # [ 135.868915] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:05.802Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"892ea19e81c57dd66993d4ca","created_at":"2026-07-06T10:10:09.507Z","trace_propagation_headers":{"sentry-trace":"88bc0b434ba94974ae27f47ac44808c8-93793177d95547e9","baggage":"sentry-trace_id=88bc0b434ba94974ae27f47ac44808c8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"56430837907e8788211e7df331d90171","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T10:10:09.521Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-892ea19e81c57dd66993d4ca: done: 0.298665 sec","job_status":"done","queue_duration_s":55.983157,"scheduling_latency_s":55.98326,"redis_calls":26,"redis_duration_s":0.015193,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.008611,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003939,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002643,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.117985107,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.298665,"completed_at":"2026-07-06T10:11:05.802Z","load_balancing_strategy":"primary","db_duration_s":0.091203,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 137.798650] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:11:07 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 2343 gitlab # [ 137.807898] setup-gitlab-start[2666]: ++ jq -r .access_token gitlab # [ 137.812473] setup-gitlab-start[2665]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token gitlab # [ 138.621467] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:11:08 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 794 gitlab # [ 138.629828] setup-gitlab-start[2639]: + BEARER=3a4dbc0bfdcce0852b2d35e62e13f66f689bd84e4d6a28f60e0fec2698d6a688 gitlab # [ 138.635593] setup-gitlab-start[2639]: + AUTH='Authorization: Bearer 3a4dbc0bfdcce0852b2d35e62e13f66f689bd84e4d6a28f60e0fec2698d6a688' gitlab # [ 138.642237] setup-gitlab-start[2639]: + curl -fs -X PUT -H 'Authorization: Bearer 3a4dbc0bfdcce0852b2d35e62e13f66f689bd84e4d6a28f60e0fec2698d6a688' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true' gitlab # [ 139.996874] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:11:09 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 1354 gitlab # [ 140.001813] setup-gitlab-start[2639]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear gitlab # [ 140.016404] runuser[2690]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 140.377357] setup-gitlab-start[2691]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.385905] setup-gitlab-start[2691]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.394530] setup-gitlab-start[2691]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.955724] setup-gitlab-start[2691]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.958498] setup-gitlab-start[2691]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.961127] setup-gitlab-start[2691]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.913590] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:19.845Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ff78c57d5272c4ea7e8e50b1","created_at":"2026-07-06T10:11:19.806Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2aa5d72eea3b078b75d6db7ab9fc12e5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:19.842Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ff78c57d5272c4ea7e8e50b1: start","job_status":"start","queue_duration_s":0.002852,"scheduling_latency_s":0.002913} gitlab # [ 149.939164] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:19.872Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ff78c57d5272c4ea7e8e50b1","created_at":"2026-07-06T10:11:19.806Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2aa5d72eea3b078b75d6db7ab9fc12e5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:19.842Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ff78c57d5272c4ea7e8e50b1: done: 0.027317 sec","job_status":"done","queue_duration_s":0.002852,"scheduling_latency_s":0.002913,"redis_calls":10,"redis_duration_s":0.003652,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000636,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00069,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000588064,"duration_s":0.027317,"completed_at":"2026-07-06T10:11:19.872Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003767,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.053576] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:19.987Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c7e8e3e0b9e2e378777cd048","created_at":"2026-07-06T10:11:19.975Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"891e948854866dc5b0f614cd561d281c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:19.984Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c7e8e3e0b9e2e378777cd048: start","job_status":"start","queue_duration_s":0.001958,"scheduling_latency_s":0.002018} gitlab # [ 150.628699] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:20.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c7e8e3e0b9e2e378777cd048","created_at":"2026-07-06T10:11:19.975Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"891e948854866dc5b0f614cd561d281c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:19.984Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c7e8e3e0b9e2e378777cd048: done: 0.575637 sec","job_status":"done","queue_duration_s":0.001958,"scheduling_latency_s":0.002018,"redis_calls":14,"redis_duration_s":0.01035,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002342,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003849,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004159,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.221256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001736533,"exclusive_lock_hold_duration_s":0.490499771,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.575637,"completed_at":"2026-07-06T10:11:20.562Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.698715] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:20.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"97f8a5f35334f6a209d48373","created_at":"2026-07-06T10:11:20.030Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.048Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-97f8a5f35334f6a209d48373: start","job_status":"start","queue_duration_s":0.584305,"scheduling_latency_s":0.58434} gitlab # [ 150.842379] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:20.775Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"97f8a5f35334f6a209d48373","created_at":"2026-07-06T10:11:20.030Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.048Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-97f8a5f35334f6a209d48373: done: 0.143498 sec","job_status":"done","queue_duration_s":0.584305,"scheduling_latency_s":0.58434,"redis_calls":71,"redis_duration_s":0.033908,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.005548,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001679,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.02559,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000570185,"duration_s":0.143498,"completed_at":"2026-07-06T10:11:20.775Z","load_balancing_strategy":"primary","db_duration_s":0.004035,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.845320] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:20.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c2e49ab2aa314f33002f99e6","created_at":"2026-07-06T10:11:20.055Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"288fbd72960b0a027cc53c2b9d426246","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.064Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c2e49ab2aa314f33002f99e6: start","job_status":"start","queue_duration_s":0.713018,"scheduling_latency_s":0.71304} gitlab # [ 151.062203] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:20.995Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c2e49ab2aa314f33002f99e6","created_at":"2026-07-06T10:11:20.055Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"288fbd72960b0a027cc53c2b9d426246","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.064Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c2e49ab2aa314f33002f99e6: done: 0.217425 sec","job_status":"done","queue_duration_s":0.713018,"scheduling_latency_s":0.71304,"redis_calls":12,"redis_duration_s":0.003458,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001255,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.042,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.147029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000443911,"exclusive_lock_hold_duration_s":0.195864279,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.217425,"completed_at":"2026-07-06T10:11:20.995Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.066263] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"acc7ca1a250f0dea2fe55c09","created_at":"2026-07-06T10:11:20.102Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00277e3c4ac0f97aa3ed3f6d3af81ba7","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.130Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-acc7ca1a250f0dea2fe55c09: start","job_status":"start","queue_duration_s":0.870165,"scheduling_latency_s":0.870187} gitlab # [ 151.099288] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"acc7ca1a250f0dea2fe55c09","created_at":"2026-07-06T10:11:20.102Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00277e3c4ac0f97aa3ed3f6d3af81ba7","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.130Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-acc7ca1a250f0dea2fe55c09: done: 0.032993 sec","job_status":"done","queue_duration_s":0.870165,"scheduling_latency_s":0.870187,"redis_calls":10,"redis_duration_s":0.005391,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001024,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003357,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000421283,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.032993,"completed_at":"2026-07-06T10:11:21.033Z","load_balancing_strategy":"primary","db_duration_s":0.008265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.170525] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.038Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4a9f73473c6e88d560d069ab","created_at":"2026-07-06T10:11:20.143Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e05c0e7f683bc248d7bf7a37ebc7292","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.149Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4a9f73473c6e88d560d069ab: start","job_status":"start","queue_duration_s":0.889091,"scheduling_latency_s":0.889115} gitlab # [ 151.552685] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.486Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4a9f73473c6e88d560d069ab","created_at":"2026-07-06T10:11:20.143Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e05c0e7f683bc248d7bf7a37ebc7292","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.149Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4a9f73473c6e88d560d069ab: done: 0.447533 sec","job_status":"done","queue_duration_s":0.889091,"scheduling_latency_s":0.889115,"redis_calls":574,"redis_duration_s":0.169944,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000454,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.167577,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.320834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000514032,"duration_s":0.447533,"completed_at":"2026-07-06T10:11:21.486Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.649256] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"584595244341c4d03e47e538","created_at":"2026-07-06T10:11:20.154Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5ac3fd7734b0021025531c4125a3d5d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.159Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-584595244341c4d03e47e538: start","job_status":"start","queue_duration_s":1.421933,"scheduling_latency_s":1.421999} gitlab # [ 151.696141] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.630Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 151.712989] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.646Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"584595244341c4d03e47e538","created_at":"2026-07-06T10:11:20.154Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5ac3fd7734b0021025531c4125a3d5d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.159Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-584595244341c4d03e47e538: done: 0.065664 sec","job_status":"done","queue_duration_s":1.421933,"scheduling_latency_s":1.421999,"redis_calls":10,"redis_duration_s":0.008391,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000849,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006404,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000556774,"duration_s":0.065664,"completed_at":"2026-07-06T10:11:21.646Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000715,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.781771] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"ba1a707919e34ccfe620a216","created_at":"2026-07-06T10:11:20.163Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25c38663472fb93f720250adc0b3b4c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.168Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-ba1a707919e34ccfe620a216: start","job_status":"start","queue_duration_s":1.547141,"scheduling_latency_s":1.547208} gitlab # [ 151.831357] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"ba1a707919e34ccfe620a216","created_at":"2026-07-06T10:11:20.163Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25c38663472fb93f720250adc0b3b4c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.168Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-ba1a707919e34ccfe620a216: done: 0.049603 sec","job_status":"done","queue_duration_s":1.547141,"scheduling_latency_s":1.547208,"redis_calls":12,"redis_duration_s":0.007304,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002208,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003872,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000948165,"duration_s":0.049603,"completed_at":"2026-07-06T10:11:21.764Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001843,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.898620] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.832Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"b1a9f906f285df87e6104d03","created_at":"2026-07-06T10:11:20.175Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.181Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-b1a9f906f285df87e6104d03: start","job_status":"start","queue_duration_s":1.650563,"scheduling_latency_s":1.650627} gitlab # [ 151.998741] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:21.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"b1a9f906f285df87e6104d03","created_at":"2026-07-06T10:11:20.175Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.181Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-b1a9f906f285df87e6104d03: done: 0.100215 sec","job_status":"done","queue_duration_s":1.650563,"scheduling_latency_s":1.650627,"redis_calls":28,"redis_duration_s":0.016624,"redis_read_bytes":36,"redis_write_bytes":10593,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002827,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.001199,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8305,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.00892,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003678,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001161601,"duration_s":0.100215,"completed_at":"2026-07-06T10:11:21.932Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.095176] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.027Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"922a0fa54cc5dd261c7e8765","created_at":"2026-07-06T10:11:20.188Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3e9c4e5428515ba3a7833cf222072c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T10:11:20.194Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-922a0fa54cc5dd261c7e8765: start","job_status":"start","queue_duration_s":1.833183,"scheduling_latency_s":1.833269} gitlab # [ 152.162788] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.095Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"922a0fa54cc5dd261c7e8765","created_at":"2026-07-06T10:11:20.188Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3e9c4e5428515ba3a7833cf222072c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T10:11:20.194Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-922a0fa54cc5dd261c7e8765: done: 0.068713 sec","job_status":"done","queue_duration_s":1.833183,"scheduling_latency_s":1.833269,"redis_calls":10,"redis_duration_s":0.0060149999999999995,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001132,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003707,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000970794,"duration_s":0.068713,"completed_at":"2026-07-06T10:11:22.095Z","load_balancing_strategy":"primary","db_duration_s":0.006669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.168301] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.102Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"80a55d40f56808bfa36b4aa7","created_at":"2026-07-06T10:11:20.208Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77d4365718f5a9b886eb0591e6a6c009","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.215Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-80a55d40f56808bfa36b4aa7: start","job_status":"start","queue_duration_s":1.88622,"scheduling_latency_s":1.886261} gitlab # [ 152.218347] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.152Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"80a55d40f56808bfa36b4aa7","created_at":"2026-07-06T10:11:20.208Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77d4365718f5a9b886eb0591e6a6c009","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.215Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-80a55d40f56808bfa36b4aa7: done: 0.050143 sec","job_status":"done","queue_duration_s":1.88622,"scheduling_latency_s":1.886261,"redis_calls":13,"redis_duration_s":0.008212,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001045,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004381,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002786,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.014675048999999999,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.050143,"completed_at":"2026-07-06T10:11:22.152Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007583,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 152.322142] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.159Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"96e68c26b373b65975629d73","created_at":"2026-07-06T10:11:20.224Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45d69a351cd073e85763f6da8a9ff0d4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.231Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-96e68c26b373b65975629d73: start","job_status":"start","queue_duration_s":1.92774,"scheduling_latency_s":1.927762} gitlab # [ 152.323769] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.199Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"96e68c26b373b65975629d73","created_at":"2026-07-06T10:11:20.224Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45d69a351cd073e85763f6da8a9ff0d4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.231Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-96e68c26b373b65975629d73: done: 0.040558 sec","job_status":"done","queue_duration_s":1.92774,"scheduling_latency_s":1.927762,"redis_calls":16,"redis_duration_s":0.00817,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003686,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337473,"duration_s":0.040558,"completed_at":"2026-07-06T10:11:22.199Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.395439] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.201Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3916124c1cd9ec578f3de061","created_at":"2026-07-06T10:11:20.253Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e97e04858da301fa6c1d766b713ce12d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T10:11:20.260Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3916124c1cd9ec578f3de061: start","job_status":"start","queue_duration_s":1.940968,"scheduling_latency_s":1.940988} gitlab # [ 152.396782] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.212Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 152.397895] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.218Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3916124c1cd9ec578f3de061","created_at":"2026-07-06T10:11:20.253Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e97e04858da301fa6c1d766b713ce12d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T10:11:20.260Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3916124c1cd9ec578f3de061: done: 0.017777 sec","job_status":"done","queue_duration_s":1.940968,"scheduling_latency_s":1.940988,"redis_calls":10,"redis_duration_s":0.00333,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00077,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001935,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000430502,"duration_s":0.017777,"completed_at":"2026-07-06T10:11:22.218Z","load_balancing_strategy":"primary","db_duration_s":0.000534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.479946] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3199088b71b8ed52c5022524","created_at":"2026-07-06T10:11:20.266Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4de32e2bd9d6a2063aa130a8d2dc83e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.272Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3199088b71b8ed52c5022524: start","job_status":"start","queue_duration_s":1.947847,"scheduling_latency_s":1.947868} gitlab # [ 152.480916] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.232Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 152.482106] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3199088b71b8ed52c5022524","created_at":"2026-07-06T10:11:20.266Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4de32e2bd9d6a2063aa130a8d2dc83e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.272Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3199088b71b8ed52c5022524: done: 0.018271 sec","job_status":"done","queue_duration_s":1.947847,"scheduling_latency_s":1.947868,"redis_calls":10,"redis_duration_s":0.0033429999999999996,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000677,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002025,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000641,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000433854,"duration_s":0.018271,"completed_at":"2026-07-06T10:11:22.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.562527] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.240Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"fa21c9dfecd1da6efef9b8ad","created_at":"2026-07-06T10:11:20.277Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6320ed37e784590205ca0c15939d657d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T10:11:20.282Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-fa21c9dfecd1da6efef9b8ad: start","job_status":"start","queue_duration_s":1.957526,"scheduling_latency_s":1.957546} gitlab # [ 152.563779] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.255Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 152.564416] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"fa21c9dfecd1da6efef9b8ad","created_at":"2026-07-06T10:11:20.277Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6320ed37e784590205ca0c15939d657d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T10:11:20.282Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-fa21c9dfecd1da6efef9b8ad: done: 0.028794 sec","job_status":"done","queue_duration_s":1.957526,"scheduling_latency_s":1.957546,"redis_calls":10,"redis_duration_s":0.00824,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003076,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003985,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001179,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00126748,"duration_s":0.028794,"completed_at":"2026-07-06T10:11:22.269Z","load_balancing_strategy":"primary","db_duration_s":0.000769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.650346] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.272Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8537ba3ed5117c94b5d32fd0","created_at":"2026-07-06T10:11:20.290Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b355579a8f14d584f55ead2183fb84ed","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.296Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8537ba3ed5117c94b5d32fd0: start","job_status":"start","queue_duration_s":1.975073,"scheduling_latency_s":1.975113} gitlab # [ 152.652330] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8537ba3ed5117c94b5d32fd0","created_at":"2026-07-06T10:11:20.290Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b355579a8f14d584f55ead2183fb84ed","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.296Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8537ba3ed5117c94b5d32fd0: done: 0.026222 sec","job_status":"done","queue_duration_s":1.975073,"scheduling_latency_s":1.975113,"redis_calls":12,"redis_duration_s":0.002873,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000447,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000497829,"exclusive_lock_hold_duration_s":0.003966426,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.026222,"completed_at":"2026-07-06T10:11:22.298Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.720939] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.299Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0ae720ea6de6e7bfaafcce10","created_at":"2026-07-06T10:11:20.305Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97c328d9655b2b3e2e6061cac562d77c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.312Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0ae720ea6de6e7bfaafcce10: start","job_status":"start","queue_duration_s":1.987541,"scheduling_latency_s":1.987562} gitlab # [ 152.724820] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.309Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 152.729836] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0ae720ea6de6e7bfaafcce10","created_at":"2026-07-06T10:11:20.305Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97c328d9655b2b3e2e6061cac562d77c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.312Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0ae720ea6de6e7bfaafcce10: done: 0.016377 sec","job_status":"done","queue_duration_s":1.987541,"scheduling_latency_s":1.987562,"redis_calls":10,"redis_duration_s":0.002085,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000319594,"duration_s":0.016377,"completed_at":"2026-07-06T10:11:22.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.796893] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1dc81e25e596b3e7b448b012","created_at":"2026-07-06T10:11:20.320Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba9ac9e20be92b9c15f3bf14009cd938","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.330Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1dc81e25e596b3e7b448b012: start","job_status":"start","queue_duration_s":1.986653,"scheduling_latency_s":1.986672} gitlab # [ 152.871185] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.339Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1dc81e25e596b3e7b448b012","created_at":"2026-07-06T10:11:20.320Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba9ac9e20be92b9c15f3bf14009cd938","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.330Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1dc81e25e596b3e7b448b012: done: 0.022123 sec","job_status":"done","queue_duration_s":1.986653,"scheduling_latency_s":1.986672,"redis_calls":10,"redis_duration_s":0.006737,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001329778,"duration_s":0.022123,"completed_at":"2026-07-06T10:11:22.339Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000368,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.939114] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"ac7576c5ed60c2b8eee43d01","created_at":"2026-07-06T10:11:20.341Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.353Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-ac7576c5ed60c2b8eee43d01: start","job_status":"start","queue_duration_s":1.987623,"scheduling_latency_s":1.987644} gitlab # [ 152.940284] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"ac7576c5ed60c2b8eee43d01","created_at":"2026-07-06T10:11:20.341Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.353Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-ac7576c5ed60c2b8eee43d01: done: 0.02736 sec","job_status":"done","queue_duration_s":1.987623,"scheduling_latency_s":1.987644,"redis_calls":19,"redis_duration_s":0.004279,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000924,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000283,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002365,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000707,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000374069,"duration_s":0.02736,"completed_at":"2026-07-06T10:11:22.368Z","load_balancing_strategy":"primary","db_duration_s":0.001547,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.012809] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.369Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b07582414c8fe46f30f4dced","created_at":"2026-07-06T10:11:20.360Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1f7d4eb72dac854e0675449f3c33c52","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.367Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b07582414c8fe46f30f4dced: start","job_status":"start","queue_duration_s":2.002124,"scheduling_latency_s":2.002143} gitlab # [ 153.014209] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b07582414c8fe46f30f4dced","created_at":"2026-07-06T10:11:20.360Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1f7d4eb72dac854e0675449f3c33c52","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.367Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b07582414c8fe46f30f4dced: done: 0.034512 sec","job_status":"done","queue_duration_s":2.002124,"scheduling_latency_s":2.002143,"redis_calls":12,"redis_duration_s":0.002797,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000503695,"exclusive_lock_hold_duration_s":0.012577577000000001,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.034512,"completed_at":"2026-07-06T10:11:22.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.110286] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f2f86501127d540a5e87d62e","created_at":"2026-07-06T10:11:20.373Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1266a5598bcb54a961a95bf7016f2be6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.378Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f2f86501127d540a5e87d62e: start","job_status":"start","queue_duration_s":2.027494,"scheduling_latency_s":2.027516} gitlab # [ 153.111413] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.422Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 153.112410] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f2f86501127d540a5e87d62e","created_at":"2026-07-06T10:11:20.373Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1266a5598bcb54a961a95bf7016f2be6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.378Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f2f86501127d540a5e87d62e: done: 0.022724 sec","job_status":"done","queue_duration_s":2.027494,"scheduling_latency_s":2.027516,"redis_calls":10,"redis_duration_s":0.0047350000000000005,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00288,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000514,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000351162,"duration_s":0.022724,"completed_at":"2026-07-06T10:11:22.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.192451] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"397a255c9ca54d0ba99c5efa","created_at":"2026-07-06T10:11:20.383Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f16319a5cb0f9cdfa0e71d8b1d7e2428","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.389Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-397a255c9ca54d0ba99c5efa: start","job_status":"start","queue_duration_s":2.040897,"scheduling_latency_s":2.040918} gitlab # [ 153.193793] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"397a255c9ca54d0ba99c5efa","created_at":"2026-07-06T10:11:20.383Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f16319a5cb0f9cdfa0e71d8b1d7e2428","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.389Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-397a255c9ca54d0ba99c5efa: done: 0.02103 sec","job_status":"done","queue_duration_s":2.040897,"scheduling_latency_s":2.040918,"redis_calls":12,"redis_duration_s":0.002921,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000437765,"exclusive_lock_hold_duration_s":0.004494146,"duration_s":0.02103,"completed_at":"2026-07-06T10:11:22.450Z","load_balancing_strategy":"primary","db_duration_s":0.002224,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.263946] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e026ab3cfca8f56edf3f0d59","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e026ab3cfca8f56edf3f0d59: start","job_status":"start","queue_duration_s":1.686653,"scheduling_latency_s":1.686673} gitlab # [ 153.265044] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e026ab3cfca8f56edf3f0d59","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e026ab3cfca8f56edf3f0d59: done: 0.020071 sec","job_status":"done","queue_duration_s":1.686653,"scheduling_latency_s":1.686673,"redis_calls":16,"redis_duration_s":0.003692,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002136,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000321828,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020071,"completed_at":"2026-07-06T10:11:22.472Z","load_balancing_strategy":"primary","db_duration_s":0.000352,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.332524] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20b585b4e4ecc49bf3c84755","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20b585b4e4ecc49bf3c84755: start","job_status":"start","queue_duration_s":1.708145,"scheduling_latency_s":1.708165} gitlab # [ 153.335514] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20b585b4e4ecc49bf3c84755","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20b585b4e4ecc49bf3c84755: done: 0.015247 sec","job_status":"done","queue_duration_s":1.708145,"scheduling_latency_s":1.708165,"redis_calls":13,"redis_duration_s":0.0027559999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015247,"completed_at":"2026-07-06T10:11:22.489Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.429468] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38fa449c140cc4a19f5f08a8","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38fa449c140cc4a19f5f08a8: start","job_status":"start","queue_duration_s":1.724786,"scheduling_latency_s":1.724807} gitlab # [ 153.431531] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38fa449c140cc4a19f5f08a8","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38fa449c140cc4a19f5f08a8: done: 0.015508 sec","job_status":"done","queue_duration_s":1.724786,"scheduling_latency_s":1.724807,"redis_calls":13,"redis_duration_s":0.003307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001292,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015508,"completed_at":"2026-07-06T10:11:22.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.474226] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4930b3c893de42231def28e","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4930b3c893de42231def28e: start","job_status":"start","queue_duration_s":1.741786,"scheduling_latency_s":1.741807} gitlab # [ 153.493544] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4930b3c893de42231def28e","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4930b3c893de42231def28e: done: 0.015409 sec","job_status":"done","queue_duration_s":1.741786,"scheduling_latency_s":1.741807,"redis_calls":13,"redis_duration_s":0.003016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015409,"completed_at":"2026-07-06T10:11:22.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.579425] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8494804d4646e66c1a1ecfc6","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8494804d4646e66c1a1ecfc6: start","job_status":"start","queue_duration_s":1.758428,"scheduling_latency_s":1.758449} gitlab # [ 153.580986] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8494804d4646e66c1a1ecfc6","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8494804d4646e66c1a1ecfc6: done: 0.016361 sec","job_status":"done","queue_duration_s":1.758428,"scheduling_latency_s":1.758449,"redis_calls":13,"redis_duration_s":0.003767,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016361,"completed_at":"2026-07-06T10:11:22.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.667248] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a53f74e4be1dbf93ae17b6f6","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a53f74e4be1dbf93ae17b6f6: start","job_status":"start","queue_duration_s":1.776003,"scheduling_latency_s":1.776024} gitlab # [ 153.668921] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a53f74e4be1dbf93ae17b6f6","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a53f74e4be1dbf93ae17b6f6: done: 0.015072 sec","job_status":"done","queue_duration_s":1.776003,"scheduling_latency_s":1.776024,"redis_calls":13,"redis_duration_s":0.002915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015072,"completed_at":"2026-07-06T10:11:22.556Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.772505] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a23d008ba6628f6398960d71","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a23d008ba6628f6398960d71: start","job_status":"start","queue_duration_s":1.792297,"scheduling_latency_s":1.792318} gitlab # [ 153.773855] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a23d008ba6628f6398960d71","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a23d008ba6628f6398960d71: done: 0.0151 sec","job_status":"done","queue_duration_s":1.792297,"scheduling_latency_s":1.792318,"redis_calls":13,"redis_duration_s":0.0027340000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0151,"completed_at":"2026-07-06T10:11:22.573Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.841373] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1290a77716bc31b76b028bc","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1290a77716bc31b76b028bc: start","job_status":"start","queue_duration_s":1.808655,"scheduling_latency_s":1.808675} gitlab # [ 153.842878] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1290a77716bc31b76b028bc","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1290a77716bc31b76b028bc: done: 0.018724 sec","job_status":"done","queue_duration_s":1.808655,"scheduling_latency_s":1.808675,"redis_calls":13,"redis_duration_s":0.006702,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005736,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018724,"completed_at":"2026-07-06T10:11:22.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.863820] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6698c5ebf4da3590de6342f7","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6698c5ebf4da3590de6342f7: start","job_status":"start","queue_duration_s":1.828655,"scheduling_latency_s":1.828676} gitlab # [ 153.882944] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6698c5ebf4da3590de6342f7","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6698c5ebf4da3590de6342f7: done: 0.01504 sec","job_status":"done","queue_duration_s":1.828655,"scheduling_latency_s":1.828676,"redis_calls":13,"redis_duration_s":0.002725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01504,"completed_at":"2026-07-06T10:11:22.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.957287] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"792f27e7f2bc8e5a00c11500","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-792f27e7f2bc8e5a00c11500: start","job_status":"start","queue_duration_s":1.844964,"scheduling_latency_s":1.844984} gitlab # [ 153.962053] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"792f27e7f2bc8e5a00c11500","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-792f27e7f2bc8e5a00c11500: done: 0.017434 sec","job_status":"done","queue_duration_s":1.844964,"scheduling_latency_s":1.844984,"redis_calls":13,"redis_duration_s":0.005182,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017434,"completed_at":"2026-07-06T10:11:22.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.037918] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"067b6de1f9d576b1a6c5a20c","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-067b6de1f9d576b1a6c5a20c: start","job_status":"start","queue_duration_s":1.86593,"scheduling_latency_s":1.865952} gitlab # [ 154.040163] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"067b6de1f9d576b1a6c5a20c","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-067b6de1f9d576b1a6c5a20c: done: 0.020425 sec","job_status":"done","queue_duration_s":1.86593,"scheduling_latency_s":1.865952,"redis_calls":13,"redis_duration_s":0.007909,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003411,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004498,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020425,"completed_at":"2026-07-06T10:11:22.652Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.114482] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"472630c6011b641c9e3ab736","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-472630c6011b641c9e3ab736: start","job_status":"start","queue_duration_s":1.887976,"scheduling_latency_s":1.887996} gitlab # [ 154.118106] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"472630c6011b641c9e3ab736","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-472630c6011b641c9e3ab736: done: 0.01804 sec","job_status":"done","queue_duration_s":1.887976,"scheduling_latency_s":1.887996,"redis_calls":13,"redis_duration_s":0.005703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002275,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003428,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01804,"completed_at":"2026-07-06T10:11:22.671Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.197143] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"272f4cf5ae2a12671376f61f","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-272f4cf5ae2a12671376f61f: start","job_status":"start","queue_duration_s":1.907371,"scheduling_latency_s":1.907392} gitlab # [ 154.198442] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"272f4cf5ae2a12671376f61f","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-272f4cf5ae2a12671376f61f: done: 0.014906 sec","job_status":"done","queue_duration_s":1.907371,"scheduling_latency_s":1.907392,"redis_calls":13,"redis_duration_s":0.00275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014906,"completed_at":"2026-07-06T10:11:22.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.267883] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a426c27b2098e2a030b97e5","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a426c27b2098e2a030b97e5: start","job_status":"start","queue_duration_s":1.923701,"scheduling_latency_s":1.923721} gitlab # [ 154.335233] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a426c27b2098e2a030b97e5","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a426c27b2098e2a030b97e5: done: 0.015032 sec","job_status":"done","queue_duration_s":1.923701,"scheduling_latency_s":1.923721,"redis_calls":13,"redis_duration_s":0.002701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001846,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015032,"completed_at":"2026-07-06T10:11:22.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.337230] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2a3a28a929597bdad1fa9b2","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2a3a28a929597bdad1fa9b2: start","job_status":"start","queue_duration_s":1.940089,"scheduling_latency_s":1.940109} gitlab # [ 154.352284] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2a3a28a929597bdad1fa9b2","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2a3a28a929597bdad1fa9b2: done: 0.01494 sec","job_status":"done","queue_duration_s":1.940089,"scheduling_latency_s":1.940109,"redis_calls":13,"redis_duration_s":0.002757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01494,"completed_at":"2026-07-06T10:11:22.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.421152] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"393a876b0eee17082c289825","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-393a876b0eee17082c289825: start","job_status":"start","queue_duration_s":1.956386,"scheduling_latency_s":1.956408} gitlab # [ 154.430394] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"393a876b0eee17082c289825","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-393a876b0eee17082c289825: done: 0.020795 sec","job_status":"done","queue_duration_s":1.956386,"scheduling_latency_s":1.956408,"redis_calls":13,"redis_duration_s":0.008522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007687,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020795,"completed_at":"2026-07-06T10:11:22.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.509832] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb6138a9542cf3c8e0fd6d84","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb6138a9542cf3c8e0fd6d84: start","job_status":"start","queue_duration_s":1.97841,"scheduling_latency_s":1.978432} gitlab # [ 154.511207] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb6138a9542cf3c8e0fd6d84","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb6138a9542cf3c8e0fd6d84: done: 0.01527 sec","job_status":"done","queue_duration_s":1.97841,"scheduling_latency_s":1.978432,"redis_calls":13,"redis_duration_s":0.002719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01527,"completed_at":"2026-07-06T10:11:22.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.579228] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48b98d4ec0b6889afdc3f6bf","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48b98d4ec0b6889afdc3f6bf: start","job_status":"start","queue_duration_s":1.994922,"scheduling_latency_s":1.994943} gitlab # [ 154.649402] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48b98d4ec0b6889afdc3f6bf","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48b98d4ec0b6889afdc3f6bf: done: 0.014979 sec","job_status":"done","queue_duration_s":1.994922,"scheduling_latency_s":1.994943,"redis_calls":13,"redis_duration_s":0.00273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001874,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014979,"completed_at":"2026-07-06T10:11:22.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.651071] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32610c072fd39ee0626b4e91","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32610c072fd39ee0626b4e91: start","job_status":"start","queue_duration_s":2.011138,"scheduling_latency_s":2.011159} gitlab # [ 154.721184] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32610c072fd39ee0626b4e91","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32610c072fd39ee0626b4e91: done: 0.015118 sec","job_status":"done","queue_duration_s":2.011138,"scheduling_latency_s":2.011159,"redis_calls":13,"redis_duration_s":0.002718,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015118,"completed_at":"2026-07-06T10:11:22.792Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.723772] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41b18cf74645d329f7dc630a","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41b18cf74645d329f7dc630a: start","job_status":"start","queue_duration_s":2.027479,"scheduling_latency_s":2.027546} gitlab # [ 154.792199] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41b18cf74645d329f7dc630a","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41b18cf74645d329f7dc630a: done: 0.016584 sec","job_status":"done","queue_duration_s":2.027479,"scheduling_latency_s":2.027546,"redis_calls":13,"redis_duration_s":0.004103,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016584,"completed_at":"2026-07-06T10:11:22.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.865128] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"560835654ceb9ef09b4a06fd","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-560835654ceb9ef09b4a06fd: start","job_status":"start","queue_duration_s":2.045428,"scheduling_latency_s":2.045449} gitlab # [ 154.866236] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"560835654ceb9ef09b4a06fd","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-560835654ceb9ef09b4a06fd: done: 0.014948 sec","job_status":"done","queue_duration_s":2.045428,"scheduling_latency_s":2.045449,"redis_calls":13,"redis_duration_s":0.002791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014948,"completed_at":"2026-07-06T10:11:22.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.938226] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0bcb762a7acfffe1c841b5d","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0bcb762a7acfffe1c841b5d: start","job_status":"start","queue_duration_s":2.061803,"scheduling_latency_s":2.061823} gitlab # [ 154.939996] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0bcb762a7acfffe1c841b5d","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0bcb762a7acfffe1c841b5d: done: 0.015179 sec","job_status":"done","queue_duration_s":2.061803,"scheduling_latency_s":2.061823,"redis_calls":13,"redis_duration_s":0.002947,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015179,"completed_at":"2026-07-06T10:11:22.842Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.013473] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eac1a3babb9c8e7c92e9337a","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eac1a3babb9c8e7c92e9337a: start","job_status":"start","queue_duration_s":2.078312,"scheduling_latency_s":2.078333} gitlab # [ 155.015469] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eac1a3babb9c8e7c92e9337a","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eac1a3babb9c8e7c92e9337a: done: 0.014921 sec","job_status":"done","queue_duration_s":2.078312,"scheduling_latency_s":2.078333,"redis_calls":13,"redis_duration_s":0.002685,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014921,"completed_at":"2026-07-06T10:11:22.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.037526] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2afb75deb7e97a730177981c","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2afb75deb7e97a730177981c: start","job_status":"start","queue_duration_s":2.094628,"scheduling_latency_s":2.094649} gitlab # [ 155.057222] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2afb75deb7e97a730177981c","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2afb75deb7e97a730177981c: done: 0.01602 sec","job_status":"done","queue_duration_s":2.094628,"scheduling_latency_s":2.094649,"redis_calls":13,"redis_duration_s":0.0031720000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01602,"completed_at":"2026-07-06T10:11:22.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.133586] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc1be469cb90d4e8534a2df0","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc1be469cb90d4e8534a2df0: start","job_status":"start","queue_duration_s":2.112092,"scheduling_latency_s":2.112112} gitlab # [ 155.200193] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc1be469cb90d4e8534a2df0","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc1be469cb90d4e8534a2df0: done: 0.015184 sec","job_status":"done","queue_duration_s":2.112092,"scheduling_latency_s":2.112112,"redis_calls":13,"redis_duration_s":0.003002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015184,"completed_at":"2026-07-06T10:11:22.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.202194] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3d555a9b46be9f2b9e7d658","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3d555a9b46be9f2b9e7d658: start","job_status":"start","queue_duration_s":2.128713,"scheduling_latency_s":2.128734} gitlab # [ 155.274359] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3d555a9b46be9f2b9e7d658","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3d555a9b46be9f2b9e7d658: done: 0.014911 sec","job_status":"done","queue_duration_s":2.128713,"scheduling_latency_s":2.128734,"redis_calls":13,"redis_duration_s":0.002731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001864,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014911,"completed_at":"2026-07-06T10:11:22.909Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.276141] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c72a62d80ec1c57d1ce4e453","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c72a62d80ec1c57d1ce4e453: start","job_status":"start","queue_duration_s":2.144843,"scheduling_latency_s":2.144863} gitlab # [ 155.348256] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c72a62d80ec1c57d1ce4e453","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c72a62d80ec1c57d1ce4e453: done: 0.017142 sec","job_status":"done","queue_duration_s":2.144843,"scheduling_latency_s":2.144863,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017142,"completed_at":"2026-07-06T10:11:22.927Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.351209] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1214f97c2004247b908f444","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1214f97c2004247b908f444: start","job_status":"start","queue_duration_s":2.163382,"scheduling_latency_s":2.163408} gitlab # [ 155.430317] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1214f97c2004247b908f444","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1214f97c2004247b908f444: done: 0.015536 sec","job_status":"done","queue_duration_s":2.163382,"scheduling_latency_s":2.163408,"redis_calls":13,"redis_duration_s":0.003044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015536,"completed_at":"2026-07-06T10:11:22.944Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.432071] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c7c401ab72829f68034bcc6","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c7c401ab72829f68034bcc6: start","job_status":"start","queue_duration_s":2.180328,"scheduling_latency_s":2.180356} gitlab # [ 155.505788] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c7c401ab72829f68034bcc6","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c7c401ab72829f68034bcc6: done: 0.016481 sec","job_status":"done","queue_duration_s":2.180328,"scheduling_latency_s":2.180356,"redis_calls":13,"redis_duration_s":0.003195,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00225,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016481,"completed_at":"2026-07-06T10:11:22.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.508301] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"887e5e3c8ebace221abe1df3","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-887e5e3c8ebace221abe1df3: start","job_status":"start","queue_duration_s":2.198158,"scheduling_latency_s":2.198179} gitlab # [ 155.528530] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"887e5e3c8ebace221abe1df3","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-887e5e3c8ebace221abe1df3: done: 0.016015 sec","job_status":"done","queue_duration_s":2.198158,"scheduling_latency_s":2.198179,"redis_calls":13,"redis_duration_s":0.0032830000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002396,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016015,"completed_at":"2026-07-06T10:11:22.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.610507] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39eac3e6157f38f22b6f885c","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39eac3e6157f38f22b6f885c: start","job_status":"start","queue_duration_s":2.215494,"scheduling_latency_s":2.215555} gitlab # [ 155.612224] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39eac3e6157f38f22b6f885c","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39eac3e6157f38f22b6f885c: done: 0.015686 sec","job_status":"done","queue_duration_s":2.215494,"scheduling_latency_s":2.215555,"redis_calls":13,"redis_duration_s":0.0032300000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002318,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015686,"completed_at":"2026-07-06T10:11:22.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.668878] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:22.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8742d4f32896b039269a166","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8742d4f32896b039269a166: start","job_status":"start","queue_duration_s":2.232425,"scheduling_latency_s":2.232446} gitlab # [ 155.687805] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8742d4f32896b039269a166","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8742d4f32896b039269a166: done: 0.015175 sec","job_status":"done","queue_duration_s":2.232425,"scheduling_latency_s":2.232446,"redis_calls":13,"redis_duration_s":0.002739,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015175,"completed_at":"2026-07-06T10:11:23.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.798164] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d3a074e6f8db4584973c031","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d3a074e6f8db4584973c031: start","job_status":"start","queue_duration_s":2.248959,"scheduling_latency_s":2.248979} gitlab # [ 155.799259] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d3a074e6f8db4584973c031","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d3a074e6f8db4584973c031: done: 0.014979 sec","job_status":"done","queue_duration_s":2.248959,"scheduling_latency_s":2.248979,"redis_calls":13,"redis_duration_s":0.0026969999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001841,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014979,"completed_at":"2026-07-06T10:11:23.029Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.824854] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59bb841dd57b2515c59909c1","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59bb841dd57b2515c59909c1: start","job_status":"start","queue_duration_s":2.265452,"scheduling_latency_s":2.265472} gitlab # [ 155.843906] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59bb841dd57b2515c59909c1","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59bb841dd57b2515c59909c1: done: 0.19573 sec","job_status":"done","queue_duration_s":2.265452,"scheduling_latency_s":2.265472,"redis_calls":13,"redis_duration_s":0.00666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002844,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003816,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.19573,"completed_at":"2026-07-06T10:11:23.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.904307] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30ec41333e2ed5efd507c396","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30ec41333e2ed5efd507c396: start","job_status":"start","queue_duration_s":2.466151,"scheduling_latency_s":2.466178} gitlab # [ 155.973461] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30ec41333e2ed5efd507c396","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30ec41333e2ed5efd507c396: done: 0.033617 sec","job_status":"done","queue_duration_s":2.466151,"scheduling_latency_s":2.466178,"redis_calls":13,"redis_duration_s":0.008578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007237,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033617,"completed_at":"2026-07-06T10:11:23.265Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 155.982432] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb61dfa52ea8d0a9d0c87cca","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb61dfa52ea8d0a9d0c87cca: start","job_status":"start","queue_duration_s":2.503687,"scheduling_latency_s":2.503709} gitlab # [ 156.059891] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb61dfa52ea8d0a9d0c87cca","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb61dfa52ea8d0a9d0c87cca: done: 0.02384 sec","job_status":"done","queue_duration_s":2.503687,"scheduling_latency_s":2.503709,"redis_calls":13,"redis_duration_s":0.010924,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007305,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02384,"completed_at":"2026-07-06T10:11:23.293Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.062397] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8824643d9e75b338917b22c","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8824643d9e75b338917b22c: start","job_status":"start","queue_duration_s":2.53095,"scheduling_latency_s":2.530972} gitlab # [ 156.081498] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8824643d9e75b338917b22c","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8824643d9e75b338917b22c: done: 0.014828 sec","job_status":"done","queue_duration_s":2.53095,"scheduling_latency_s":2.530972,"redis_calls":13,"redis_duration_s":0.002794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014828,"completed_at":"2026-07-06T10:11:23.311Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.167237] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"988f7ad6ab8ffa7ef41b46da","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-988f7ad6ab8ffa7ef41b46da: start","job_status":"start","queue_duration_s":2.547081,"scheduling_latency_s":2.547102} gitlab # [ 156.169209] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"988f7ad6ab8ffa7ef41b46da","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-988f7ad6ab8ffa7ef41b46da: done: 0.015826 sec","job_status":"done","queue_duration_s":2.547081,"scheduling_latency_s":2.547102,"redis_calls":13,"redis_duration_s":0.004013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001179,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002834,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015826,"completed_at":"2026-07-06T10:11:23.328Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.259561] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28d0a9e1a3d38ab91435714a","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28d0a9e1a3d38ab91435714a: start","job_status":"start","queue_duration_s":2.564585,"scheduling_latency_s":2.564605} gitlab # [ 156.261160] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28d0a9e1a3d38ab91435714a","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28d0a9e1a3d38ab91435714a: done: 0.016869 sec","job_status":"done","queue_duration_s":2.564585,"scheduling_latency_s":2.564605,"redis_calls":13,"redis_duration_s":0.004866,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001429,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003437,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016869,"completed_at":"2026-07-06T10:11:23.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.332444] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8934b8ce62ad36278ccb061","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8934b8ce62ad36278ccb061: start","job_status":"start","queue_duration_s":2.583147,"scheduling_latency_s":2.583168} gitlab # [ 156.334249] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8934b8ce62ad36278ccb061","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8934b8ce62ad36278ccb061: done: 0.020559 sec","job_status":"done","queue_duration_s":2.583147,"scheduling_latency_s":2.583168,"redis_calls":13,"redis_duration_s":0.008150000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020559,"completed_at":"2026-07-06T10:11:23.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.408566] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.373Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca1dafa19f9c09d9e0f78346","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca1dafa19f9c09d9e0f78346: start","job_status":"start","queue_duration_s":2.607107,"scheduling_latency_s":2.607128} gitlab # [ 156.410568] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca1dafa19f9c09d9e0f78346","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca1dafa19f9c09d9e0f78346: done: 0.022525 sec","job_status":"done","queue_duration_s":2.607107,"scheduling_latency_s":2.607128,"redis_calls":13,"redis_duration_s":0.00978,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007153,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022525,"completed_at":"2026-07-06T10:11:23.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.506601] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9663f7fd1f03cb8251cbdd25","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9663f7fd1f03cb8251cbdd25: start","job_status":"start","queue_duration_s":2.632879,"scheduling_latency_s":2.632901} gitlab # [ 156.508574] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9663f7fd1f03cb8251cbdd25","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9663f7fd1f03cb8251cbdd25: done: 0.021905 sec","job_status":"done","queue_duration_s":2.632879,"scheduling_latency_s":2.632901,"redis_calls":13,"redis_duration_s":0.009091,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00307,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021905,"completed_at":"2026-07-06T10:11:23.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.605799] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b49e244d719a6a3d0196a5a1","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b49e244d719a6a3d0196a5a1: start","job_status":"start","queue_duration_s":2.656545,"scheduling_latency_s":2.656566} gitlab # [ 156.607703] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b49e244d719a6a3d0196a5a1","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b49e244d719a6a3d0196a5a1: done: 0.015829 sec","job_status":"done","queue_duration_s":2.656545,"scheduling_latency_s":2.656566,"redis_calls":13,"redis_duration_s":0.003225,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015829,"completed_at":"2026-07-06T10:11:23.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.621354] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"699fdd354d791d929ac4e0f4","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-699fdd354d791d929ac4e0f4: start","job_status":"start","queue_duration_s":2.673725,"scheduling_latency_s":2.673746} gitlab # [ 156.642508] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"699fdd354d791d929ac4e0f4","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-699fdd354d791d929ac4e0f4: done: 0.015693 sec","job_status":"done","queue_duration_s":2.673725,"scheduling_latency_s":2.673746,"redis_calls":13,"redis_duration_s":0.003083,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015693,"completed_at":"2026-07-06T10:11:23.455Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.764604] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a381c84d02997c77f512ba23","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a381c84d02997c77f512ba23: start","job_status":"start","queue_duration_s":2.690982,"scheduling_latency_s":2.691003} gitlab # [ 156.766499] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a381c84d02997c77f512ba23","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a381c84d02997c77f512ba23: done: 0.015699 sec","job_status":"done","queue_duration_s":2.690982,"scheduling_latency_s":2.691003,"redis_calls":13,"redis_duration_s":0.003012,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015699,"completed_at":"2026-07-06T10:11:23.472Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.781223] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c32029c270d3cbffce10b9d","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c32029c270d3cbffce10b9d: start","job_status":"start","queue_duration_s":2.70819,"scheduling_latency_s":2.70821} gitlab # [ 156.801256] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c32029c270d3cbffce10b9d","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c32029c270d3cbffce10b9d: done: 0.015325 sec","job_status":"done","queue_duration_s":2.70819,"scheduling_latency_s":2.70821,"redis_calls":13,"redis_duration_s":0.00298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015325,"completed_at":"2026-07-06T10:11:23.489Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.901380] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b035144a3480def81b2468f","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b035144a3480def81b2468f: start","job_status":"start","queue_duration_s":2.724955,"scheduling_latency_s":2.724975} gitlab # [ 156.903043] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b035144a3480def81b2468f","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b035144a3480def81b2468f: done: 0.016016 sec","job_status":"done","queue_duration_s":2.724955,"scheduling_latency_s":2.724975,"redis_calls":13,"redis_duration_s":0.002981,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016016,"completed_at":"2026-07-06T10:11:23.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 156.985532] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"527a54097db6ed855d8672f3","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-527a54097db6ed855d8672f3: start","job_status":"start","queue_duration_s":2.742339,"scheduling_latency_s":2.74236} gitlab # [ 156.987190] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"527a54097db6ed855d8672f3","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-527a54097db6ed855d8672f3: done: 0.017178 sec","job_status":"done","queue_duration_s":2.742339,"scheduling_latency_s":2.74236,"redis_calls":13,"redis_duration_s":0.004446,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017178,"completed_at":"2026-07-06T10:11:23.525Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.019176] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85a0f8e8d6018597bbb0f091","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85a0f8e8d6018597bbb0f091: start","job_status":"start","queue_duration_s":2.761068,"scheduling_latency_s":2.761089} gitlab # [ 157.089850] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85a0f8e8d6018597bbb0f091","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85a0f8e8d6018597bbb0f091: done: 0.015969 sec","job_status":"done","queue_duration_s":2.761068,"scheduling_latency_s":2.761089,"redis_calls":13,"redis_duration_s":0.0031049999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015969,"completed_at":"2026-07-06T10:11:23.542Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.099441] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41bdf63d7c330feb52b6fdd1","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41bdf63d7c330feb52b6fdd1: start","job_status":"start","queue_duration_s":2.77831,"scheduling_latency_s":2.77833} gitlab # [ 157.184377] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41bdf63d7c330feb52b6fdd1","created_at":"2026-07-06T10:11:20.683Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-c19bdd08e56249d2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b88a619dc4fe1bd2bec17827d77e86e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:20.765Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41bdf63d7c330feb52b6fdd1: done: 0.015575 sec","job_status":"done","queue_duration_s":2.77831,"scheduling_latency_s":2.77833,"redis_calls":13,"redis_duration_s":0.00304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015575,"completed_at":"2026-07-06T10:11:23.559Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.186575] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"dd0e4b68aed473de9959552d","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-dd0e4b68aed473de9959552d: start","job_status":"start","queue_duration_s":1.645794,"scheduling_latency_s":1.645815} gitlab # [ 157.200762] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"dd0e4b68aed473de9959552d","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-dd0e4b68aed473de9959552d: done: 0.037312 sec","job_status":"done","queue_duration_s":1.645794,"scheduling_latency_s":1.645815,"redis_calls":16,"redis_duration_s":0.004266,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000693,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002492,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000343899,"duration_s":0.037312,"completed_at":"2026-07-06T10:11:23.598Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.315481] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f8f6f25d25e5668bf36c58f7","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f8f6f25d25e5668bf36c58f7: start","job_status":"start","queue_duration_s":1.684592,"scheduling_latency_s":1.684612} gitlab # [ 157.317047] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f8f6f25d25e5668bf36c58f7","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f8f6f25d25e5668bf36c58f7: done: 0.019347 sec","job_status":"done","queue_duration_s":1.684592,"scheduling_latency_s":1.684612,"redis_calls":13,"redis_duration_s":0.00382,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019347,"completed_at":"2026-07-06T10:11:23.619Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.389678] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4f21a759b2a532c3f15baef6","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4f21a759b2a532c3f15baef6: start","job_status":"start","queue_duration_s":1.705299,"scheduling_latency_s":1.70532} gitlab # [ 157.391446] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4f21a759b2a532c3f15baef6","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4f21a759b2a532c3f15baef6: done: 0.015839 sec","job_status":"done","queue_duration_s":1.705299,"scheduling_latency_s":1.70532,"redis_calls":13,"redis_duration_s":0.003223,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015839,"completed_at":"2026-07-06T10:11:23.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.429239] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6f59a063da92403ba5507ed9","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6f59a063da92403ba5507ed9: start","job_status":"start","queue_duration_s":1.722689,"scheduling_latency_s":1.722711} gitlab # [ 157.450211] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6f59a063da92403ba5507ed9","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6f59a063da92403ba5507ed9: done: 0.016235 sec","job_status":"done","queue_duration_s":1.722689,"scheduling_latency_s":1.722711,"redis_calls":13,"redis_duration_s":0.0034790000000000003,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016235,"completed_at":"2026-07-06T10:11:23.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.531450] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c9a5d6d9ab52d83e3f0ca6a4","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c9a5d6d9ab52d83e3f0ca6a4: start","job_status":"start","queue_duration_s":1.74041,"scheduling_latency_s":1.74043} gitlab # [ 157.533170] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c9a5d6d9ab52d83e3f0ca6a4","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c9a5d6d9ab52d83e3f0ca6a4: done: 0.01552 sec","job_status":"done","queue_duration_s":1.74041,"scheduling_latency_s":1.74043,"redis_calls":13,"redis_duration_s":0.002981,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01552,"completed_at":"2026-07-06T10:11:23.671Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000704,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.605558] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"461736fbe9e9834001b93e48","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-461736fbe9e9834001b93e48: start","job_status":"start","queue_duration_s":1.757259,"scheduling_latency_s":1.757279} gitlab # [ 157.610731] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"461736fbe9e9834001b93e48","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-461736fbe9e9834001b93e48: done: 0.016016 sec","job_status":"done","queue_duration_s":1.757259,"scheduling_latency_s":1.757279,"redis_calls":13,"redis_duration_s":0.0030640000000000003,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016016,"completed_at":"2026-07-06T10:11:23.688Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.712581] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"af05ea4752eb737b19090085","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-af05ea4752eb737b19090085: start","job_status":"start","queue_duration_s":1.774674,"scheduling_latency_s":1.774695} gitlab # [ 157.714563] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"af05ea4752eb737b19090085","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-af05ea4752eb737b19090085: done: 0.014924 sec","job_status":"done","queue_duration_s":1.774674,"scheduling_latency_s":1.774695,"redis_calls":13,"redis_duration_s":0.002875,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001828,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014924,"completed_at":"2026-07-06T10:11:23.704Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.754688] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7e17c961183f734a0fd81281","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7e17c961183f734a0fd81281: start","job_status":"start","queue_duration_s":1.791561,"scheduling_latency_s":1.791582} gitlab # [ 157.830640] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7e17c961183f734a0fd81281","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7e17c961183f734a0fd81281: done: 0.014918 sec","job_status":"done","queue_duration_s":1.791561,"scheduling_latency_s":1.791582,"redis_calls":13,"redis_duration_s":0.0028780000000000003,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014918,"completed_at":"2026-07-06T10:11:23.721Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.834313] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6e9da69c134f6ea6c9d2e6ff","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6e9da69c134f6ea6c9d2e6ff: start","job_status":"start","queue_duration_s":1.807752,"scheduling_latency_s":1.807773} gitlab # [ 157.855489] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6e9da69c134f6ea6c9d2e6ff","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6e9da69c134f6ea6c9d2e6ff: done: 0.015011 sec","job_status":"done","queue_duration_s":1.807752,"scheduling_latency_s":1.807773,"redis_calls":13,"redis_duration_s":0.002713,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015011,"completed_at":"2026-07-06T10:11:23.737Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.953256] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"cdd8fc824c09ee8f9405b409","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-cdd8fc824c09ee8f9405b409: start","job_status":"start","queue_duration_s":1.824274,"scheduling_latency_s":1.824295} gitlab # [ 157.954701] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"cdd8fc824c09ee8f9405b409","created_at":"2026-07-06T10:11:21.876Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-8cc21075e88c46d4","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"fe566ad5e0c76afaee538445b357b635","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:21.915Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-cdd8fc824c09ee8f9405b409: done: 0.014849 sec","job_status":"done","queue_duration_s":1.824274,"scheduling_latency_s":1.824295,"redis_calls":13,"redis_duration_s":0.002823,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014849,"completed_at":"2026-07-06T10:11:23.754Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000465,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 157.999947] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.755Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e2a926fec4c23d821e8faf5e","created_at":"2026-07-06T10:11:22.353Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-1c5d3b363bf94ee2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:22.362Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e2a926fec4c23d821e8faf5e: start","job_status":"start","queue_duration_s":1.393476,"scheduling_latency_s":1.393497} gitlab # [ 158.074115] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.769Z","message":"No existing merge request to be cleaned up."} gitlab # [ 158.075708] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.783Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e2a926fec4c23d821e8faf5e","created_at":"2026-07-06T10:11:22.353Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-1c5d3b363bf94ee2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:22.362Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e2a926fec4c23d821e8faf5e: done: 0.028032 sec","job_status":"done","queue_duration_s":1.393476,"scheduling_latency_s":1.393497,"redis_calls":13,"redis_duration_s":0.007555,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002376,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004736,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001033092,"duration_s":0.028032,"completed_at":"2026-07-06T10:11:23.783Z","load_balancing_strategy":"primary","db_duration_s":0.002986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.083494] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.785Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a74ca52261df359cad77e492","created_at":"2026-07-06T10:11:22.353Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-1c5d3b363bf94ee2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:22.362Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a74ca52261df359cad77e492: start","job_status":"start","queue_duration_s":1.423066,"scheduling_latency_s":1.423087} gitlab # [ 158.104315] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.797Z","message":"No existing merge request to be cleaned up."} gitlab # [ 158.106865] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.804Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a74ca52261df359cad77e492","created_at":"2026-07-06T10:11:22.353Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-1c5d3b363bf94ee2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:22.362Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a74ca52261df359cad77e492: done: 0.019169 sec","job_status":"done","queue_duration_s":1.423066,"scheduling_latency_s":1.423087,"redis_calls":10,"redis_duration_s":0.002543,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001521,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019169,"completed_at":"2026-07-06T10:11:23.804Z","load_balancing_strategy":"primary","db_duration_s":0.006089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.184434] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.805Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"35da742af459e10374b54d59","created_at":"2026-07-06T10:11:22.353Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-1c5d3b363bf94ee2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:22.362Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-35da742af459e10374b54d59: start","job_status":"start","queue_duration_s":1.443614,"scheduling_latency_s":1.443635} gitlab # [ 158.185980] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.816Z","message":"No existing merge request to be cleaned up."} gitlab # [ 158.187623] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.823Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"35da742af459e10374b54d59","created_at":"2026-07-06T10:11:22.353Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-1c5d3b363bf94ee2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:22.362Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-35da742af459e10374b54d59: done: 0.017316 sec","job_status":"done","queue_duration_s":1.443614,"scheduling_latency_s":1.443635,"redis_calls":10,"redis_duration_s":0.002385,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001409,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017316,"completed_at":"2026-07-06T10:11:23.823Z","load_balancing_strategy":"primary","db_duration_s":0.003302,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.292547] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.824Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5e9de8a2c28a010a08c6cca5","created_at":"2026-07-06T10:11:22.353Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-1c5d3b363bf94ee2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:22.362Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5e9de8a2c28a010a08c6cca5: start","job_status":"start","queue_duration_s":1.462558,"scheduling_latency_s":1.462578} gitlab # [ 158.293800] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.834Z","message":"No existing merge request to be cleaned up."} gitlab # [ 158.294719] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:11:23.840Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5e9de8a2c28a010a08c6cca5","created_at":"2026-07-06T10:11:22.353Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-1c5d3b363bf94ee2","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85eba3717389a8996f23d58d7e98b0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:11:22.362Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5e9de8a2c28a010a08c6cca5: done: 0.015645 sec","job_status":"done","queue_duration_s":1.462558,"scheduling_latency_s":1.462578,"redis_calls":10,"redis_duration_s":0.002185,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001328,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015645,"completed_at":"2026-07-06T10:11:23.840Z","load_balancing_strategy":"primary","db_duration_s":0.001508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.653053] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 6120999248 wd_nsec: 6120998875 gitlab # [ 167.486680] setup-gitlab-start[2691]: W, [2026-07-06T10:11:37.419935 #2691] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 190.673059] runuser[2690]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 190.675776] setup-gitlab-start[2903]: +++ date -d '+7 days' +%F gitlab # [ 190.679264] setup-gitlab-start[2902]: ++ jq -r .token gitlab # [ 190.681375] setup-gitlab-start[2901]: ++ curl -fs -X POST -H 'Authorization: Bearer 3a4dbc0bfdcce0852b2d35e62e13f66f689bd84e4d6a28f60e0fec2698d6a688' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-13"}' http://gitlab/api/v4/users/1/personal_access_tokens gitlab # [ 190.993682] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:00.926Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"bd51762eb4c0c81aeb47d736","created_at":"2026-07-06T10:12:00.899Z","trace_propagation_headers":{"sentry-trace":"143fd74c70d942feb1991a75c89d40a3-ab4aed4f7f764553","baggage":"sentry-trace_id=143fd74c70d942feb1991a75c89d40a3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKKZCPXGWG49TX528EN3Q","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:2f00f5c363334f1952d8a1f06fcfca9c32c947c3cd9d07b928321c990afbcbcd","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:00.923Z","job_size_bytes":451,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-bd51762eb4c0c81aeb47d736: start","job_status":"start","queue_duration_s":0.003068,"scheduling_latency_s":0.003103} gitlab # [ 191.004636] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:00 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 317 gitlab # [ 191.009265] setup-gitlab-start[2639]: + TOKEN=glpat-8oMi3V19JrFfoPAb8qHhVm86MQp1OjEH.01.0w05133df gitlab # [ 191.011422] setup-gitlab-start[2639]: + test -n glpat-8oMi3V19JrFfoPAb8qHhVm86MQp1OjEH.01.0w05133df gitlab # [ 191.014123] setup-gitlab-start[2639]: + test glpat-8oMi3V19JrFfoPAb8qHhVm86MQp1OjEH.01.0w05133df '!=' null gitlab # [ 191.016893] setup-gitlab-start[2639]: + mkdir -p /var/lib/secrets gitlab # [ 191.019133] setup-gitlab-start[2639]: + echo glpat-8oMi3V19JrFfoPAb8qHhVm86MQp1OjEH.01.0w05133df gitlab # [ 191.021848] setup-gitlab-start[2639]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 191.025354] setup-gitlab-start[2639]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-8oMi3V19JrFfoPAb8qHhVm86MQp1OjEH.01.0w05133df' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 191.198425] sidekiq-cluster[2108]: {"severity":"WARN","time":"2026-07-06T10:12:01.131Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"bd51762eb4c0c81aeb47d736","created_at":"2026-07-06T10:12:00.899Z","trace_propagation_headers":{"sentry-trace":"143fd74c70d942feb1991a75c89d40a3-ab4aed4f7f764553","baggage":"sentry-trace_id=143fd74c70d942feb1991a75c89d40a3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKKZCPXGWG49TX528EN3Q","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:2f00f5c363334f1952d8a1f06fcfca9c32c947c3cd9d07b928321c990afbcbcd","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:00.923Z","job_size_bytes":451,"pid":2108,"sidekiq_tid":"lyoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-bd51762eb4c0c81aeb47d736: fail: 0.205288 sec","job_status":"fail","queue_duration_s":0.003068,"scheduling_latency_s":0.003103,"redis_calls":11,"redis_duration_s":0.004318,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001013,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000460394,"duration_s":0.205288,"completed_at":"2026-07-06T10:12:01.131Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.036309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.267748] gitaly[1385]: time="2026-07-06T10:12:02.200Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.200" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.271342] gitaly[1385]: time="2026-07-06T10:12:02.201Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.200" grpc.time_ms=0.977 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.463045] gitaly[1385]: time="2026-07-06T10:12:02.396Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.206" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.396" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.464957] gitaly[1385]: time="2026-07-06T10:12:02.397Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.206" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.396" grpc.time_ms=0.236 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.501923] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:02.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8c20db196d8af8368814b748","created_at":"2026-07-06T10:12:02.427Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eef549fb631bc4b4984af3b1c9ea536f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.432Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8c20db196d8af8368814b748: start","job_status":"start","queue_duration_s":0.002118,"scheduling_latency_s":0.002145} gitlab # [ 192.560315] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:02.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8c20db196d8af8368814b748","created_at":"2026-07-06T10:12:02.427Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eef549fb631bc4b4984af3b1c9ea536f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.432Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8c20db196d8af8368814b748: done: 0.058609 sec","job_status":"done","queue_duration_s":0.002118,"scheduling_latency_s":0.002145,"redis_calls":7,"redis_duration_s":0.024847,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.020325,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058609,"completed_at":"2026-07-06T10:12:02.493Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005273,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.567638] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:02.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"90bbfcf87e15a1957330d063","created_at":"2026-07-06T10:12:02.456Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ba63374f4799204d485646b48f34a91","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.468Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-90bbfcf87e15a1957330d063: start","job_status":"start","queue_duration_s":0.027236,"scheduling_latency_s":0.02726} gitlab # [ 192.683298] gitaly[1385]: time="2026-07-06T10:12:02.617Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T diskcache=b54ab513-ebfd-4700-9730-f9a24aa806f3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T10:12:12.001" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.582" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.684480] gitaly[1385]: time="2026-07-06T10:12:02.617Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=32 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T10:12:12.001" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.582" grpc.time_ms=35.412 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.720982] gitaly[1385]: time="2026-07-06T10:12:02.655Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.074" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.654" pid=1385 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 192.723307] gitaly[1385]: time="2026-07-06T10:12:02.655Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.074" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.654" grpc.time_ms=0.452 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.764251] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:02.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"90bbfcf87e15a1957330d063","created_at":"2026-07-06T10:12:02.456Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ba63374f4799204d485646b48f34a91","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.468Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-90bbfcf87e15a1957330d063: done: 0.200931 sec","job_status":"done","queue_duration_s":0.027236,"scheduling_latency_s":0.02726,"redis_calls":9,"redis_duration_s":0.043316,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.04084,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002476,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000500902,"exclusive_lock_hold_duration_s":0.135692665,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.200931,"completed_at":"2026-07-06T10:12:02.696Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.12587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.770659] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:02.704Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"716cf06127278161f774ed64","created_at":"2026-07-06T10:12:02.481Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f48074e169e97ed3cac3c29dbf047367","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.584Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-716cf06127278161f774ed64: start","job_status":"start","queue_duration_s":0.119284,"scheduling_latency_s":0.119343} gitlab # [ 192.859486] gitaly[1385]: time="2026-07-06T10:12:02.793Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=59 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=97 command.spawn_token_fork_ms=0 command.system_time_ms=49 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:12:12.114" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.694" grpc.time_ms=98.716 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.865701] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:02.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"716cf06127278161f774ed64","created_at":"2026-07-06T10:12:02.481Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f48074e169e97ed3cac3c29dbf047367","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.584Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-716cf06127278161f774ed64: done: 0.095192 sec","job_status":"done","queue_duration_s":0.119284,"scheduling_latency_s":0.119343,"redis_calls":12,"redis_duration_s":0.013045,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005447,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005269,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002119264,"exclusive_lock_hold_duration_s":0.009131328000000001,"duration_s":0.095192,"completed_at":"2026-07-06T10:12:02.799Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.877740] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:02.806Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"dc41dc6c6b9b611a906eadd5","created_at":"2026-07-06T10:12:02.593Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.595Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-dc41dc6c6b9b611a906eadd5: start","job_status":"start","queue_duration_s":0.210526,"scheduling_latency_s":0.210554} gitlab # [ 192.967254] gitaly[1385]: time="2026-07-06T10:12:02.901Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.320" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.901" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.983553] gitaly[1385]: time="2026-07-06T10:12:02.901Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.320" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.901" grpc.time_ms=0.308 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.018597] gitaly[1385]: time="2026-07-06T10:12:02.952Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T diskcache=cfc02ce5-8922-4908-b3cf-f326a2c158d6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T10:12:12.000" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.911" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.025157] gitaly[1385]: time="2026-07-06T10:12:02.959Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.048" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.959" pid=1385 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 193.026462] gitaly[1385]: time="2026-07-06T10:12:02.959Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:12.048" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.959" grpc.time_ms=0.453 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.036152] gitaly[1385]: time="2026-07-06T10:12:02.970Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:12:12.053" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.964" grpc.time_ms=5.539 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.079871] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.013Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"dc41dc6c6b9b611a906eadd5","created_at":"2026-07-06T10:12:02.593Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.595Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-dc41dc6c6b9b611a906eadd5: done: 0.207414 sec","job_status":"done","queue_duration_s":0.210526,"scheduling_latency_s":0.210554,"redis_calls":60,"redis_duration_s":0.083065,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000499,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.078048,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.207414,"completed_at":"2026-07-06T10:12:03.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.082218] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f0bda1504942186165fb5bf4","created_at":"2026-07-06T10:12:02.599Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc9eb9b936bd9d12e3c8f9e215b9e3f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.601Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f0bda1504942186165fb5bf4: start","job_status":"start","queue_duration_s":0.413899,"scheduling_latency_s":0.413924} gitlab # [ 193.171702] gitaly[1385]: time="2026-07-06T10:12:03.105Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=322 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T10:12:12.000" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:02.911" grpc.time_ms=41.844 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.197987] gitaly[1385]: time="2026-07-06T10:12:03.132Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:13.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:03.131" grpc.time_ms=0.152 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.216236] gitaly[1385]: time="2026-07-06T10:12:03.150Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T10:12:13.019" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:03.150" grpc.time_ms=0.176 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.313597] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.247Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f0bda1504942186165fb5bf4","created_at":"2026-07-06T10:12:02.599Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc9eb9b936bd9d12e3c8f9e215b9e3f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.601Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f0bda1504942186165fb5bf4: done: 0.231935 sec","job_status":"done","queue_duration_s":0.413899,"scheduling_latency_s":0.413924,"redis_calls":11,"redis_duration_s":0.003094,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00056,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001434,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.098,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.098,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00056767,"exclusive_lock_hold_duration_s":0.146424475,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.231935,"completed_at":"2026-07-06T10:12:03.247Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.115024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.358125] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.248Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"638b29151e0cbf2de86130cf","created_at":"2026-07-06T10:12:02.607Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3421d5faa6c3274d2d46b734bc02cd40","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.618Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-638b29151e0cbf2de86130cf: start","job_status":"start","queue_duration_s":0.63083,"scheduling_latency_s":0.630858} gitlab # [ 193.359338] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"638b29151e0cbf2de86130cf","created_at":"2026-07-06T10:12:02.607Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3421d5faa6c3274d2d46b734bc02cd40","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.618Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-638b29151e0cbf2de86130cf: done: 0.018275 sec","job_status":"done","queue_duration_s":0.63083,"scheduling_latency_s":0.630858,"redis_calls":12,"redis_duration_s":0.002619,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000475,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337473,"duration_s":0.018275,"completed_at":"2026-07-06T10:12:03.267Z","load_balancing_strategy":"primary","db_duration_s":0.000519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.458443] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d18534b4a627a2ef4e13136a","created_at":"2026-07-06T10:12:02.639Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a3e15bc4fca43922226804aa49c9946","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.642Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d18534b4a627a2ef4e13136a: start","job_status":"start","queue_duration_s":0.626444,"scheduling_latency_s":0.626465} gitlab # [ 193.459884] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d18534b4a627a2ef4e13136a","created_at":"2026-07-06T10:12:02.639Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a3e15bc4fca43922226804aa49c9946","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.642Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d18534b4a627a2ef4e13136a: done: 0.018033 sec","job_status":"done","queue_duration_s":0.626444,"scheduling_latency_s":0.626465,"redis_calls":7,"redis_duration_s":0.001699,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.018033,"completed_at":"2026-07-06T10:12:03.286Z","load_balancing_strategy":"primary","db_duration_s":0.004271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.530823] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"a7808a011dfab3bc12e0e04b","created_at":"2026-07-06T10:12:02.691Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ea77a0aae959b2c0df2f09a26ca8bc9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.707Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-a7808a011dfab3bc12e0e04b: start","job_status":"start","queue_duration_s":0.580207,"scheduling_latency_s":0.580231} gitlab # [ 193.532621] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.310Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"a7808a011dfab3bc12e0e04b","created_at":"2026-07-06T10:12:02.691Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ea77a0aae959b2c0df2f09a26ca8bc9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.707Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-a7808a011dfab3bc12e0e04b: done: 0.022766 sec","job_status":"done","queue_duration_s":0.580207,"scheduling_latency_s":0.580231,"redis_calls":10,"redis_duration_s":0.0022170000000000002,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001347,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335797,"duration_s":0.022766,"completed_at":"2026-07-06T10:12:03.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002889,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.581860] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.433Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4871f8c083f8ab45b4f98227","created_at":"2026-07-06T10:12:02.724Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4984fc124db395099f6f4d676aa3750a","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.728Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4871f8c083f8ab45b4f98227: start","job_status":"start","queue_duration_s":0.705013,"scheduling_latency_s":0.70508} gitlab # [ 193.724940] gitaly[1385]: time="2026-07-06T10:12:03.657Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKMA9J8BHQ9PGBFQ28A1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:12:13.065" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:03.645" grpc.time_ms=11.211 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.998707] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:03.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4871f8c083f8ab45b4f98227","created_at":"2026-07-06T10:12:02.724Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4984fc124db395099f6f4d676aa3750a","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.728Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4871f8c083f8ab45b4f98227: done: 0.499214 sec","job_status":"done","queue_duration_s":0.705013,"scheduling_latency_s":0.70508,"redis_calls":571,"redis_duration_s":0.193976,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.191805,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.344087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.499214,"completed_at":"2026-07-06T10:12:03.932Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.101244] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1cecd1833a8984acef376b28","created_at":"2026-07-06T10:12:02.741Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e85285f88910618b276760a06ed1c38","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.746Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1cecd1833a8984acef376b28: start","job_status":"start","queue_duration_s":1.288398,"scheduling_latency_s":1.288435} gitlab # [ 194.130820] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.064Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 194.141687] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1cecd1833a8984acef376b28","created_at":"2026-07-06T10:12:02.741Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e85285f88910618b276760a06ed1c38","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.746Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1cecd1833a8984acef376b28: done: 0.040529 sec","job_status":"done","queue_duration_s":1.288398,"scheduling_latency_s":1.288435,"redis_calls":7,"redis_duration_s":0.007065,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040529,"completed_at":"2026-07-06T10:12:04.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.207266] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4342a82b45c4d659ef31f324","created_at":"2026-07-06T10:12:02.753Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2ae780877dea64e28ad208177393047","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.756Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4342a82b45c4d659ef31f324: start","job_status":"start","queue_duration_s":1.383988,"scheduling_latency_s":1.384059} gitlab # [ 194.246729] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.180Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4342a82b45c4d659ef31f324","created_at":"2026-07-06T10:12:02.753Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2ae780877dea64e28ad208177393047","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.756Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4342a82b45c4d659ef31f324: done: 0.039545 sec","job_status":"done","queue_duration_s":1.383988,"scheduling_latency_s":1.384059,"redis_calls":9,"redis_duration_s":0.005041,"redis_read_bytes":7,"redis_write_bytes":1053,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001207,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":277,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039545,"completed_at":"2026-07-06T10:12:04.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.314467] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.248Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"95f499bfa1abce2c760f5b31","created_at":"2026-07-06T10:12:02.795Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"92769acdedc521329c3cc2d1ca90af97","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.802Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-95f499bfa1abce2c760f5b31: start","job_status":"start","queue_duration_s":1.445777,"scheduling_latency_s":1.445816} gitlab # [ 194.337841] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:04 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 3301 gitlab # [ 194.342563] setup-gitlab-start[2639]: + mkdir -p /tmp/test-flake gitlab # [ 194.345395] setup-gitlab-start[2639]: + cd /tmp/test-flake gitlab # [ 194.347217] setup-gitlab-start[2639]: + git init -b master gitlab # [ 194.357651] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.291Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"95f499bfa1abce2c760f5b31","created_at":"2026-07-06T10:12:02.795Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"92769acdedc521329c3cc2d1ca90af97","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.802Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-95f499bfa1abce2c760f5b31: done: 0.043104 sec","job_status":"done","queue_duration_s":1.445777,"scheduling_latency_s":1.445816,"redis_calls":10,"redis_duration_s":0.005697,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001062,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000838934,"duration_s":0.043104,"completed_at":"2026-07-06T10:12:04.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000785,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.377858] setup-gitlab-start[2951]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 194.379595] setup-gitlab-start[2639]: + git config user.name test gitlab # [ 194.382374] setup-gitlab-start[2639]: + git config user.email test@example.com gitlab # [ 194.386424] setup-gitlab-start[2639]: + cat gitlab # [ 194.389957] setup-gitlab-start[2639]: + git add flake.nix gitlab # [ 194.394954] setup-gitlab-start[2639]: + git commit -m 'initial commit' gitlab # [ 194.409712] setup-gitlab-start[2956]: [master (root-commit) 0483f69] initial commit gitlab # [ 194.411308] setup-gitlab-start[2956]: 1 file changed, 10 insertions(+) gitlab # [ 194.413824] setup-gitlab-start[2956]: create mode 100644 flake.nix gitlab # [ 194.415604] setup-gitlab-start[2639]: + git remote add origin http://oauth2:glpat-8oMi3V19JrFfoPAb8qHhVm86MQp1OjEH.01.0w05133df@gitlab/root/test-flake.git gitlab # [ 194.419868] setup-gitlab-start[2639]: + git push -u origin master gitlab # [ 194.443825] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.377Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a56bc50a2c8b092860719da4","created_at":"2026-07-06T10:12:02.816Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d27edb25940dc07c974eb0bf16a3a9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T10:12:02.818Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a56bc50a2c8b092860719da4: start","job_status":"start","queue_duration_s":1.558655,"scheduling_latency_s":1.558825} gitlab # [ 194.485121] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.417Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a56bc50a2c8b092860719da4","created_at":"2026-07-06T10:12:02.816Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d27edb25940dc07c974eb0bf16a3a9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T10:12:02.818Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a56bc50a2c8b092860719da4: done: 0.040754 sec","job_status":"done","queue_duration_s":1.558655,"scheduling_latency_s":1.558825,"redis_calls":7,"redis_duration_s":0.0044989999999999995,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040754,"completed_at":"2026-07-06T10:12:04.417Z","load_balancing_strategy":"primary","db_duration_s":0.002202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.493389] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:04 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 36 gitlab # [ 194.551494] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.484Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"57681b9226c54a3840de1bcc","created_at":"2026-07-06T10:12:02.825Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2b105465237ae0fdb851c9580a3ef14","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.837Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-57681b9226c54a3840de1bcc: start","job_status":"start","queue_duration_s":1.647704,"scheduling_latency_s":1.647743} gitlab # [ 194.570858] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"57681b9226c54a3840de1bcc","created_at":"2026-07-06T10:12:02.825Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2b105465237ae0fdb851c9580a3ef14","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.837Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-57681b9226c54a3840de1bcc: done: 0.020196 sec","job_status":"done","queue_duration_s":1.647704,"scheduling_latency_s":1.647743,"redis_calls":12,"redis_duration_s":0.00314,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000571,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001558,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000380775,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.020196,"completed_at":"2026-07-06T10:12:04.504Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.639158] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.506Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b4f43dfde576f5d13826f28b","created_at":"2026-07-06T10:12:02.809Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.837Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b4f43dfde576f5d13826f28b: start","job_status":"start","queue_duration_s":1.668867,"scheduling_latency_s":1.668887} gitlab # [ 194.643185] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.549Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b4f43dfde576f5d13826f28b","created_at":"2026-07-06T10:12:02.809Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.837Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b4f43dfde576f5d13826f28b: done: 0.043073 sec","job_status":"done","queue_duration_s":1.668867,"scheduling_latency_s":1.668887,"redis_calls":10,"redis_duration_s":0.003072,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00199,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.026671546,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.043073,"completed_at":"2026-07-06T10:12:04.549Z","load_balancing_strategy":"primary","db_duration_s":0.011156,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 194.713159] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.550Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"65574c93bce4802c41068c86","created_at":"2026-07-06T10:12:02.844Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.848Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-65574c93bce4802c41068c86: start","job_status":"start","queue_duration_s":1.70217,"scheduling_latency_s":1.702192} gitlab # [ 194.714294] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.618Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"65574c93bce4802c41068c86","created_at":"2026-07-06T10:12:02.844Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.848Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-65574c93bce4802c41068c86: done: 0.067598 sec","job_status":"done","queue_duration_s":1.70217,"scheduling_latency_s":1.702192,"redis_calls":14,"redis_duration_s":0.003711,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000753,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00173,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001228,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.047418037999999996,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.067598,"completed_at":"2026-07-06T10:12:04.618Z","load_balancing_strategy":"primary","db_duration_s":0.013021,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 194.805516] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.619Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"68a93f0506d873292ecaee56","created_at":"2026-07-06T10:12:02.848Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d97e3031e6d53d3d1918b78ebb321a4","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.853Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-68a93f0506d873292ecaee56: start","job_status":"start","queue_duration_s":1.765897,"scheduling_latency_s":1.765921} gitlab # [ 194.806877] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"68a93f0506d873292ecaee56","created_at":"2026-07-06T10:12:02.848Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d97e3031e6d53d3d1918b78ebb321a4","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.853Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-68a93f0506d873292ecaee56: done: 0.016966 sec","job_status":"done","queue_duration_s":1.765897,"scheduling_latency_s":1.765921,"redis_calls":10,"redis_duration_s":0.002334,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001358,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002867403,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.016966,"completed_at":"2026-07-06T10:12:04.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001382,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 194.879912] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.638Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"465502aaebb205e0fe15b919","created_at":"2026-07-06T10:12:02.863Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"385a01e78e861d02da887e6f6ff76fff","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.866Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-465502aaebb205e0fe15b919: start","job_status":"start","queue_duration_s":1.771652,"scheduling_latency_s":1.771674} gitlab # [ 194.884478] gitaly[1385]: time="2026-07-06T10:12:04.624Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQPDVTKM4NS4XGGDMHHK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:04.612" grpc.time_ms=11.617 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.892430] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:04 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 195 gitlab # [ 194.893286] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.656Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"465502aaebb205e0fe15b919","created_at":"2026-07-06T10:12:02.863Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"385a01e78e861d02da887e6f6ff76fff","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.866Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-465502aaebb205e0fe15b919: done: 0.017933 sec","job_status":"done","queue_duration_s":1.771652,"scheduling_latency_s":1.771674,"redis_calls":7,"redis_duration_s":0.001715,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00048,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017933,"completed_at":"2026-07-06T10:12:04.656Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003468,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.966240] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.657Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1264596313ff1c44ddc4fd39","created_at":"2026-07-06T10:12:02.887Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a06481cfa153333db6837a85cb7ab66","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T10:12:02.891Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1264596313ff1c44ddc4fd39: start","job_status":"start","queue_duration_s":1.766389,"scheduling_latency_s":1.766412} gitlab # [ 194.975188] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.664Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 194.981086] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1264596313ff1c44ddc4fd39","created_at":"2026-07-06T10:12:02.887Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a06481cfa153333db6837a85cb7ab66","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T10:12:02.891Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1264596313ff1c44ddc4fd39: done: 0.011816 sec","job_status":"done","queue_duration_s":1.766389,"scheduling_latency_s":1.766412,"redis_calls":7,"redis_duration_s":0.001634,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001183,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011816,"completed_at":"2026-07-06T10:12:04.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.067301] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.670Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0a6bdd7478961109d45af470","created_at":"2026-07-06T10:12:02.897Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82b3ddddd4292f8ab1dc54e96d990968","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.900Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0a6bdd7478961109d45af470: start","job_status":"start","queue_duration_s":1.769814,"scheduling_latency_s":1.769846} gitlab # [ 195.069910] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.677Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 195.071762] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0a6bdd7478961109d45af470","created_at":"2026-07-06T10:12:02.897Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82b3ddddd4292f8ab1dc54e96d990968","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.900Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0a6bdd7478961109d45af470: done: 0.011498 sec","job_status":"done","queue_duration_s":1.769814,"scheduling_latency_s":1.769846,"redis_calls":7,"redis_duration_s":0.001675,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011498,"completed_at":"2026-07-06T10:12:04.682Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.163114] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.683Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d0e45f4723dd6c1767d06c1c","created_at":"2026-07-06T10:12:02.907Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56c18e2cd330bbb96c00b644b74a52e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T10:12:02.909Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d0e45f4723dd6c1767d06c1c: start","job_status":"start","queue_duration_s":1.773715,"scheduling_latency_s":1.773734} gitlab # [ 195.164338] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.690Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 195.166108] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d0e45f4723dd6c1767d06c1c","created_at":"2026-07-06T10:12:02.907Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56c18e2cd330bbb96c00b644b74a52e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T10:12:02.909Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d0e45f4723dd6c1767d06c1c: done: 0.011657 sec","job_status":"done","queue_duration_s":1.773715,"scheduling_latency_s":1.773734,"redis_calls":7,"redis_duration_s":0.00158,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011657,"completed_at":"2026-07-06T10:12:04.694Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.246268] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d06750982f432d1d29ddd342","created_at":"2026-07-06T10:12:02.916Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbba0e16de461e601b47036df2b814ed","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.920Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d06750982f432d1d29ddd342: start","job_status":"start","queue_duration_s":1.776008,"scheduling_latency_s":1.776069} gitlab # [ 195.247771] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d06750982f432d1d29ddd342","created_at":"2026-07-06T10:12:02.916Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbba0e16de461e601b47036df2b814ed","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.920Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d06750982f432d1d29ddd342: done: 0.013613 sec","job_status":"done","queue_duration_s":1.776008,"scheduling_latency_s":1.776069,"redis_calls":9,"redis_duration_s":0.002106,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000440279,"exclusive_lock_hold_duration_s":0.0019536,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.013613,"completed_at":"2026-07-06T10:12:04.709Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000888,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.325144] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9eb354de02a89f550b1230b4","created_at":"2026-07-06T10:12:02.929Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb6d9cac7aee3326fbc88d77ac0055d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.932Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9eb354de02a89f550b1230b4: start","job_status":"start","queue_duration_s":1.778344,"scheduling_latency_s":1.778365} gitlab # [ 195.328689] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.720Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 195.334057] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.725Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9eb354de02a89f550b1230b4","created_at":"2026-07-06T10:12:02.929Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb6d9cac7aee3326fbc88d77ac0055d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.932Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9eb354de02a89f550b1230b4: done: 0.013799 sec","job_status":"done","queue_duration_s":1.778344,"scheduling_latency_s":1.778365,"redis_calls":7,"redis_duration_s":0.003762,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002461,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013799,"completed_at":"2026-07-06T10:12:04.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.429087] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d48ef08517a868d673d1b2ce","created_at":"2026-07-06T10:12:02.942Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f30662a7e8feae770a78a563c70fefea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.946Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d48ef08517a868d673d1b2ce: start","job_status":"start","queue_duration_s":1.779844,"scheduling_latency_s":1.779865} gitlab # [ 195.430137] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.738Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d48ef08517a868d673d1b2ce","created_at":"2026-07-06T10:12:02.942Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f30662a7e8feae770a78a563c70fefea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.946Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d48ef08517a868d673d1b2ce: done: 0.01182 sec","job_status":"done","queue_duration_s":1.779844,"scheduling_latency_s":1.779865,"redis_calls":7,"redis_duration_s":0.0017460000000000002,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01182,"completed_at":"2026-07-06T10:12:04.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.536197] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.739Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c70c4059a28f5eb9c9ab51bb","created_at":"2026-07-06T10:12:02.953Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.957Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c70c4059a28f5eb9c9ab51bb: start","job_status":"start","queue_duration_s":1.782394,"scheduling_latency_s":1.782415} gitlab # [ 195.537721] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c70c4059a28f5eb9c9ab51bb","created_at":"2026-07-06T10:12:02.953Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.957Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c70c4059a28f5eb9c9ab51bb: done: 0.020057 sec","job_status":"done","queue_duration_s":1.782394,"scheduling_latency_s":1.782415,"redis_calls":14,"redis_duration_s":0.0034419999999999997,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000286,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00242,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020057,"completed_at":"2026-07-06T10:12:04.759Z","load_balancing_strategy":"primary","db_duration_s":0.000833,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.608651] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ba90636dfd0551a4b648cbe7","created_at":"2026-07-06T10:12:02.965Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af87815796073241b547c3c2f20f06c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.969Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ba90636dfd0551a4b648cbe7: start","job_status":"start","queue_duration_s":1.791507,"scheduling_latency_s":1.791563} gitlab # [ 195.609826] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.786Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ba90636dfd0551a4b648cbe7","created_at":"2026-07-06T10:12:02.965Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af87815796073241b547c3c2f20f06c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.969Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ba90636dfd0551a4b648cbe7: done: 0.024725 sec","job_status":"done","queue_duration_s":1.791507,"scheduling_latency_s":1.791563,"redis_calls":9,"redis_duration_s":0.002282,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001154,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000560407,"exclusive_lock_hold_duration_s":0.007781715,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.024725,"completed_at":"2026-07-06T10:12:04.786Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.681294] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e3beac0ef4fc4296bb9c05a9","created_at":"2026-07-06T10:12:02.979Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f8d4a599463a98e7fe8eebc8e0864b9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.982Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e3beac0ef4fc4296bb9c05a9: start","job_status":"start","queue_duration_s":1.80484,"scheduling_latency_s":1.804861} gitlab # [ 195.682314] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.794Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 195.686767] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e3beac0ef4fc4296bb9c05a9","created_at":"2026-07-06T10:12:02.979Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f8d4a599463a98e7fe8eebc8e0864b9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.982Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e3beac0ef4fc4296bb9c05a9: done: 0.011324 sec","job_status":"done","queue_duration_s":1.80484,"scheduling_latency_s":1.804861,"redis_calls":7,"redis_duration_s":0.001604,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011324,"completed_at":"2026-07-06T10:12:04.798Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.757190] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4ddda8f0081243ff46d3be3c","created_at":"2026-07-06T10:12:02.989Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80e10412554c282605c8ade9e03f6c4f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.992Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4ddda8f0081243ff46d3be3c: start","job_status":"start","queue_duration_s":1.808204,"scheduling_latency_s":1.808224} gitlab # [ 195.829474] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4ddda8f0081243ff46d3be3c","created_at":"2026-07-06T10:12:02.989Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-3e87280241cb4c87","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80e10412554c282605c8ade9e03f6c4f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:02.992Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4ddda8f0081243ff46d3be3c: done: 0.01353 sec","job_status":"done","queue_duration_s":1.808204,"scheduling_latency_s":1.808224,"redis_calls":9,"redis_duration_s":0.002171,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000446705,"exclusive_lock_hold_duration_s":0.001858058,"duration_s":0.01353,"completed_at":"2026-07-06T10:12:04.813Z","load_balancing_strategy":"primary","db_duration_s":0.000664,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.832193] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5c3b6d07c6a91d8d82e6460","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5c3b6d07c6a91d8d82e6460: start","job_status":"start","queue_duration_s":1.807543,"scheduling_latency_s":1.807563} gitlab # [ 195.904491] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5c3b6d07c6a91d8d82e6460","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5c3b6d07c6a91d8d82e6460: done: 0.015737 sec","job_status":"done","queue_duration_s":1.807543,"scheduling_latency_s":1.807563,"redis_calls":13,"redis_duration_s":0.003349,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002461,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015737,"completed_at":"2026-07-06T10:12:04.830Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.912200] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1af2188034d162d731e2c9c4","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1af2188034d162d731e2c9c4: start","job_status":"start","queue_duration_s":1.824673,"scheduling_latency_s":1.824694} gitlab # [ 195.984517] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1af2188034d162d731e2c9c4","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1af2188034d162d731e2c9c4: done: 0.015014 sec","job_status":"done","queue_duration_s":1.824673,"scheduling_latency_s":1.824694,"redis_calls":13,"redis_duration_s":0.003053,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015014,"completed_at":"2026-07-06T10:12:04.847Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.991294] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb5dc00e6ddfd6d29411c856","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb5dc00e6ddfd6d29411c856: start","job_status":"start","queue_duration_s":1.841123,"scheduling_latency_s":1.841153} gitlab # [ 196.011047] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb5dc00e6ddfd6d29411c856","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb5dc00e6ddfd6d29411c856: done: 0.014662 sec","job_status":"done","queue_duration_s":1.841123,"scheduling_latency_s":1.841153,"redis_calls":13,"redis_duration_s":0.002883,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014662,"completed_at":"2026-07-06T10:12:04.863Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.083958] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a0583e68e0ff1e4c7db28c2","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a0583e68e0ff1e4c7db28c2: start","job_status":"start","queue_duration_s":1.85727,"scheduling_latency_s":1.857291} gitlab # [ 196.155103] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a0583e68e0ff1e4c7db28c2","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a0583e68e0ff1e4c7db28c2: done: 0.014553 sec","job_status":"done","queue_duration_s":1.85727,"scheduling_latency_s":1.857291,"redis_calls":13,"redis_duration_s":0.003048,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014553,"completed_at":"2026-07-06T10:12:04.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.157049] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39c6008fb0d852c6e8fe3543","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39c6008fb0d852c6e8fe3543: start","job_status":"start","queue_duration_s":1.873168,"scheduling_latency_s":1.873189} gitlab # [ 196.171049] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39c6008fb0d852c6e8fe3543","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39c6008fb0d852c6e8fe3543: done: 0.015338 sec","job_status":"done","queue_duration_s":1.873168,"scheduling_latency_s":1.873189,"redis_calls":13,"redis_duration_s":0.0030050000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015338,"completed_at":"2026-07-06T10:12:04.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.263194] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54f3d1c5c9401a8534e851ce","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54f3d1c5c9401a8534e851ce: start","job_status":"start","queue_duration_s":1.889809,"scheduling_latency_s":1.88983} gitlab # [ 196.264233] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54f3d1c5c9401a8534e851ce","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54f3d1c5c9401a8534e851ce: done: 0.020895 sec","job_status":"done","queue_duration_s":1.889809,"scheduling_latency_s":1.88983,"redis_calls":13,"redis_duration_s":0.006732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004723,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020895,"completed_at":"2026-07-06T10:12:04.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.333135] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bab33d6f8505f88912e3f20","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bab33d6f8505f88912e3f20: start","job_status":"start","queue_duration_s":1.912053,"scheduling_latency_s":1.912075} gitlab # [ 196.335339] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bab33d6f8505f88912e3f20","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bab33d6f8505f88912e3f20: done: 0.018542 sec","job_status":"done","queue_duration_s":1.912053,"scheduling_latency_s":1.912075,"redis_calls":13,"redis_duration_s":0.004551,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001018,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003533,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018542,"completed_at":"2026-07-06T10:12:04.938Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.414343] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ef368c64cfaa599063f230b","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ef368c64cfaa599063f230b: start","job_status":"start","queue_duration_s":1.932745,"scheduling_latency_s":1.932782} gitlab # [ 196.417307] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ef368c64cfaa599063f230b","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ef368c64cfaa599063f230b: done: 0.018118 sec","job_status":"done","queue_duration_s":1.932745,"scheduling_latency_s":1.932782,"redis_calls":13,"redis_duration_s":0.0036680000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002716,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018118,"completed_at":"2026-07-06T10:12:04.958Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.476751] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"518fb88adf4049b192adb7e0","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-518fb88adf4049b192adb7e0: start","job_status":"start","queue_duration_s":1.952283,"scheduling_latency_s":1.952304} gitlab # [ 196.550090] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"518fb88adf4049b192adb7e0","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-518fb88adf4049b192adb7e0: done: 0.018738 sec","job_status":"done","queue_duration_s":1.952283,"scheduling_latency_s":1.952304,"redis_calls":13,"redis_duration_s":0.00365,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018738,"completed_at":"2026-07-06T10:12:04.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.560772] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e4f21a55b01c8e44abd7b24","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e4f21a55b01c8e44abd7b24: start","job_status":"start","queue_duration_s":1.972546,"scheduling_latency_s":1.972568} gitlab # [ 196.574607] gitaly[1385]: time="2026-07-06T10:12:04.862Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:12:14.271" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:04.852" grpc.time_ms=9.839 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.595117] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e4f21a55b01c8e44abd7b24","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e4f21a55b01c8e44abd7b24: done: 0.015884 sec","job_status":"done","queue_duration_s":1.972546,"scheduling_latency_s":1.972568,"redis_calls":13,"redis_duration_s":0.00316,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002215,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015884,"completed_at":"2026-07-06T10:12:04.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.670231] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:04.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ff41238139791458a7a2f68","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ff41238139791458a7a2f68: start","job_status":"start","queue_duration_s":1.989689,"scheduling_latency_s":1.98971} gitlab: (finished: waiting for unit setup-gitlab.service, in 197.73 seconds) gitlab # [ 196.737201] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ff41238139791458a7a2f68","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ff41238139791458a7a2f68: done: 0.015012 sec","job_status":"done","queue_duration_s":1.989689,"scheduling_latency_s":1.98971,"redis_calls":13,"redis_duration_s":0.0030169999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015012,"completed_at":"2026-07-06T10:12:05.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: waiting for unit nixbot.service gitlab # [ 196.739199] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5828eb022b76047e18ee913","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5828eb022b76047e18ee913: start","job_status":"start","queue_duration_s":2.006245,"scheduling_latency_s":2.006265} gitlab # [ 196.753299] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5828eb022b76047e18ee913","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5828eb022b76047e18ee913: done: 0.021953 sec","job_status":"done","queue_duration_s":2.006245,"scheduling_latency_s":2.006265,"redis_calls":13,"redis_duration_s":0.0029389999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021953,"completed_at":"2026-07-06T10:12:05.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.860169] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1452217c32a8cb34bf93f1ad","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1452217c32a8cb34bf93f1ad: start","job_status":"start","queue_duration_s":2.029529,"scheduling_latency_s":2.02955} gitlab # [ 196.861189] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1452217c32a8cb34bf93f1ad","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1452217c32a8cb34bf93f1ad: done: 0.015039 sec","job_status":"done","queue_duration_s":2.029529,"scheduling_latency_s":2.02955,"redis_calls":13,"redis_duration_s":0.002908,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015039,"completed_at":"2026-07-06T10:12:05.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.933516] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffae1f1ef72d7ccffb2348e7","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffae1f1ef72d7ccffb2348e7: start","job_status":"start","queue_duration_s":2.045845,"scheduling_latency_s":2.045909} gitlab # [ 196.934937] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffae1f1ef72d7ccffb2348e7","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffae1f1ef72d7ccffb2348e7: done: 0.015216 sec","job_status":"done","queue_duration_s":2.045845,"scheduling_latency_s":2.045909,"redis_calls":13,"redis_duration_s":0.003065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015216,"completed_at":"2026-07-06T10:12:05.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.005777] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"880e147abd29981d6c84a80c","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-880e147abd29981d6c84a80c: start","job_status":"start","queue_duration_s":2.062404,"scheduling_latency_s":2.062425} gitlab # [ 197.007201] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"880e147abd29981d6c84a80c","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-880e147abd29981d6c84a80c: done: 0.015618 sec","job_status":"done","queue_duration_s":2.062404,"scheduling_latency_s":2.062425,"redis_calls":13,"redis_duration_s":0.00317,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015618,"completed_at":"2026-07-06T10:12:05.085Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.089202] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"818cd1a081e3ff89e93e6734","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-818cd1a081e3ff89e93e6734: start","job_status":"start","queue_duration_s":2.079352,"scheduling_latency_s":2.079373} gitlab # [ 197.090376] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"818cd1a081e3ff89e93e6734","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-818cd1a081e3ff89e93e6734: done: 0.015336 sec","job_status":"done","queue_duration_s":2.079352,"scheduling_latency_s":2.079373,"redis_calls":13,"redis_duration_s":0.002965,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015336,"completed_at":"2026-07-06T10:12:05.102Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.175133] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56f8ff244085d513d1d2bfc9","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56f8ff244085d513d1d2bfc9: start","job_status":"start","queue_duration_s":2.096122,"scheduling_latency_s":2.096152} gitlab # [ 197.176328] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56f8ff244085d513d1d2bfc9","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56f8ff244085d513d1d2bfc9: done: 0.015256 sec","job_status":"done","queue_duration_s":2.096122,"scheduling_latency_s":2.096152,"redis_calls":13,"redis_duration_s":0.003169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015256,"completed_at":"2026-07-06T10:12:05.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.253334] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae73e03b68df1c51a688b09f","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae73e03b68df1c51a688b09f: start","job_status":"start","queue_duration_s":2.112653,"scheduling_latency_s":2.112674} gitlab # [ 197.254725] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae73e03b68df1c51a688b09f","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae73e03b68df1c51a688b09f: done: 0.015081 sec","job_status":"done","queue_duration_s":2.112653,"scheduling_latency_s":2.112674,"redis_calls":13,"redis_duration_s":0.003149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015081,"completed_at":"2026-07-06T10:12:05.135Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.329902] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e97b6dcf2691281395f1e36","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e97b6dcf2691281395f1e36: start","job_status":"start","queue_duration_s":2.129106,"scheduling_latency_s":2.129127} gitlab # [ 197.331540] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e97b6dcf2691281395f1e36","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e97b6dcf2691281395f1e36: done: 0.014952 sec","job_status":"done","queue_duration_s":2.129106,"scheduling_latency_s":2.129127,"redis_calls":13,"redis_duration_s":0.002949,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014952,"completed_at":"2026-07-06T10:12:05.151Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.350758] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34bd3bccf0d4bad8461fb5d7","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34bd3bccf0d4bad8461fb5d7: start","job_status":"start","queue_duration_s":2.145358,"scheduling_latency_s":2.145379} gitlab # [ 197.417898] gitaly[1385]: time="2026-07-06T10:12:04.888Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-06T10:12:34.079" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-06T10:12:04.879" grpc.time_ms=8.513 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.419782] gitaly[1385]: time="2026-07-06T10:12:04.910Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB duration_ms=122ns method=POST pid=1385 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 197.423144] gitaly[1385]: time="2026-07-06T10:12:04.969Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB duration_ms=58ns method=POST pid=1385 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 197.424221] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34bd3bccf0d4bad8461fb5d7","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34bd3bccf0d4bad8461fb5d7: done: 0.014783 sec","job_status":"done","queue_duration_s":2.145358,"scheduling_latency_s":2.145379,"redis_calls":13,"redis_duration_s":0.003022,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014783,"completed_at":"2026-07-06T10:12:05.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.513051] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b20b31632e50dcf712ad586","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b20b31632e50dcf712ad586: start","job_status":"start","queue_duration_s":2.161501,"scheduling_latency_s":2.161523} gitlab # [ 197.514051] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b20b31632e50dcf712ad586","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b20b31632e50dcf712ad586: done: 0.021163 sec","job_status":"done","queue_duration_s":2.161501,"scheduling_latency_s":2.161523,"redis_calls":13,"redis_duration_s":0.004067,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021163,"completed_at":"2026-07-06T10:12:05.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.594409] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0067ccfe6118edd62ba6318e","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0067ccfe6118edd62ba6318e: start","job_status":"start","queue_duration_s":2.184468,"scheduling_latency_s":2.184489} gitlab # [ 197.595845] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0067ccfe6118edd62ba6318e","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0067ccfe6118edd62ba6318e: done: 0.018752 sec","job_status":"done","queue_duration_s":2.184468,"scheduling_latency_s":2.184489,"redis_calls":13,"redis_duration_s":0.003563,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002431,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018752,"completed_at":"2026-07-06T10:12:05.210Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.668206] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d42ba5a2df25d176e70f0eb","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d42ba5a2df25d176e70f0eb: start","job_status":"start","queue_duration_s":2.204417,"scheduling_latency_s":2.204438} gitlab # [ 197.669680] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d42ba5a2df25d176e70f0eb","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d42ba5a2df25d176e70f0eb: done: 0.014815 sec","job_status":"done","queue_duration_s":2.204417,"scheduling_latency_s":2.204438,"redis_calls":13,"redis_duration_s":0.002948,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014815,"completed_at":"2026-07-06T10:12:05.226Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.689208] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a49daf168bab7056c26ca50","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a49daf168bab7056c26ca50: start","job_status":"start","queue_duration_s":2.220492,"scheduling_latency_s":2.220513} gitlab # [ 197.754426] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a49daf168bab7056c26ca50","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a49daf168bab7056c26ca50: done: 0.014932 sec","job_status":"done","queue_duration_s":2.220492,"scheduling_latency_s":2.220513,"redis_calls":13,"redis_duration_s":0.003092,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014932,"completed_at":"2026-07-06T10:12:05.242Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.825336] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88d6a2655c02a6f6470b0ccf","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88d6a2655c02a6f6470b0ccf: start","job_status":"start","queue_duration_s":2.236841,"scheduling_latency_s":2.236906} gitlab # [ 197.826780] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88d6a2655c02a6f6470b0ccf","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88d6a2655c02a6f6470b0ccf: done: 0.01642 sec","job_status":"done","queue_duration_s":2.236841,"scheduling_latency_s":2.236906,"redis_calls":13,"redis_duration_s":0.003273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01642,"completed_at":"2026-07-06T10:12:05.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.903472] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c21de6906530e78cb7136b5","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c21de6906530e78cb7136b5: start","job_status":"start","queue_duration_s":2.254979,"scheduling_latency_s":2.255023} gitlab # [ 197.905137] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c21de6906530e78cb7136b5","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c21de6906530e78cb7136b5: done: 0.021321 sec","job_status":"done","queue_duration_s":2.254979,"scheduling_latency_s":2.255023,"redis_calls":13,"redis_duration_s":0.0042769999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001416,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021321,"completed_at":"2026-07-06T10:12:05.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.974284] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6fa98055be3fa63f4aa030b","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6fa98055be3fa63f4aa030b: start","job_status":"start","queue_duration_s":2.277979,"scheduling_latency_s":2.278} gitlab # [ 197.975504] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6fa98055be3fa63f4aa030b","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6fa98055be3fa63f4aa030b: done: 0.020227 sec","job_status":"done","queue_duration_s":2.277979,"scheduling_latency_s":2.278,"redis_calls":13,"redis_duration_s":0.003547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001173,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002374,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020227,"completed_at":"2026-07-06T10:12:05.305Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.050963] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6d3afc4c7918ed5ef6e5d50","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6d3afc4c7918ed5ef6e5d50: start","job_status":"start","queue_duration_s":2.299675,"scheduling_latency_s":2.299703} gitlab # [ 198.051949] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6d3afc4c7918ed5ef6e5d50","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6d3afc4c7918ed5ef6e5d50: done: 0.015458 sec","job_status":"done","queue_duration_s":2.299675,"scheduling_latency_s":2.299703,"redis_calls":13,"redis_duration_s":0.00311,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015458,"completed_at":"2026-07-06T10:12:05.322Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.128808] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"559b76cdb4601d8a83d7d8ac","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-559b76cdb4601d8a83d7d8ac: start","job_status":"start","queue_duration_s":2.316577,"scheduling_latency_s":2.316597} gitlab # [ 198.130113] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"559b76cdb4601d8a83d7d8ac","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-559b76cdb4601d8a83d7d8ac: done: 0.01615 sec","job_status":"done","queue_duration_s":2.316577,"scheduling_latency_s":2.316597,"redis_calls":13,"redis_duration_s":0.0034929999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001161,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002332,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01615,"completed_at":"2026-07-06T10:12:05.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.207865] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c1c472afc50937bcb312f75","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c1c472afc50937bcb312f75: start","job_status":"start","queue_duration_s":2.334108,"scheduling_latency_s":2.33413} gitlab # [ 198.209623] gitlab-workhorse[1287]: time="2026-07-06T10:12:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 198.211186] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:06 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 1913 gitlab # [ 198.212553] systemd[1]: Finished setup-gitlab.service. gitlab # [ 198.213555] setup-gitlab-start[2960]: To http://gitlab/root/test-flake.git gitlab # [ 198.214632] setup-gitlab-start[2960]: * [new branch] master -> master gitlab # [ 198.215902] setup-gitlab-start[2960]: branch 'master' set up to track 'origin/master'. gitlab # [ 198.217471] gitaly[1385]: time="2026-07-06T10:12:04.970Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5321 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:04.787" grpc.time_ms=182.606 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.218296] gitaly[1385]: time="2026-07-06T10:12:05.059Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:05.059" grpc.time_ms=0.529 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.219405] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c1c472afc50937bcb312f75","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c1c472afc50937bcb312f75: done: 0.015304 sec","job_status":"done","queue_duration_s":2.334108,"scheduling_latency_s":2.33413,"redis_calls":13,"redis_duration_s":0.0031360000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015304,"completed_at":"2026-07-06T10:12:05.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.292567] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b034020d35dd2d74f1bbed41","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b034020d35dd2d74f1bbed41: start","job_status":"start","queue_duration_s":2.350701,"scheduling_latency_s":2.350721} gitlab # [ 198.296392] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.373Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b034020d35dd2d74f1bbed41","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b034020d35dd2d74f1bbed41: done: 0.015026 sec","job_status":"done","queue_duration_s":2.350701,"scheduling_latency_s":2.350721,"redis_calls":13,"redis_duration_s":0.003186,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015026,"completed_at":"2026-07-06T10:12:05.373Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.381217] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573d111fe56509ddaa03a49b","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573d111fe56509ddaa03a49b: start","job_status":"start","queue_duration_s":2.367098,"scheduling_latency_s":2.367119} gitlab # [ 198.382404] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573d111fe56509ddaa03a49b","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573d111fe56509ddaa03a49b: done: 0.014949 sec","job_status":"done","queue_duration_s":2.367098,"scheduling_latency_s":2.367119,"redis_calls":13,"redis_duration_s":0.00311,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002222,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014949,"completed_at":"2026-07-06T10:12:05.389Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.473117] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67b098e0e27e05008d600931","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67b098e0e27e05008d600931: start","job_status":"start","queue_duration_s":2.383592,"scheduling_latency_s":2.383613} gitlab # [ 198.475540] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67b098e0e27e05008d600931","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67b098e0e27e05008d600931: done: 0.01642 sec","job_status":"done","queue_duration_s":2.383592,"scheduling_latency_s":2.383613,"redis_calls":13,"redis_duration_s":0.0030819999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01642,"completed_at":"2026-07-06T10:12:05.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.555097] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5638311f0a62e4eb7ca3f5a8","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5638311f0a62e4eb7ca3f5a8: start","job_status":"start","queue_duration_s":2.401589,"scheduling_latency_s":2.401617} gitlab # [ 198.556358] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5638311f0a62e4eb7ca3f5a8","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5638311f0a62e4eb7ca3f5a8: done: 0.015467 sec","job_status":"done","queue_duration_s":2.401589,"scheduling_latency_s":2.401617,"redis_calls":13,"redis_duration_s":0.002759,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015467,"completed_at":"2026-07-06T10:12:05.424Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.633379] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef90b8f9a1595beff83e029a","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef90b8f9a1595beff83e029a: start","job_status":"start","queue_duration_s":2.418284,"scheduling_latency_s":2.418305} gitlab # [ 198.634844] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef90b8f9a1595beff83e029a","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef90b8f9a1595beff83e029a: done: 0.014348 sec","job_status":"done","queue_duration_s":2.418284,"scheduling_latency_s":2.418305,"redis_calls":13,"redis_duration_s":0.0027140000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001856,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014348,"completed_at":"2026-07-06T10:12:05.440Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.713472] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"435974bcd48a25773b8590b7","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-435974bcd48a25773b8590b7: start","job_status":"start","queue_duration_s":2.433907,"scheduling_latency_s":2.433928} gitlab # [ 198.714313] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"435974bcd48a25773b8590b7","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-435974bcd48a25773b8590b7: done: 0.014376 sec","job_status":"done","queue_duration_s":2.433907,"scheduling_latency_s":2.433928,"redis_calls":13,"redis_duration_s":0.002763,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014376,"completed_at":"2026-07-06T10:12:05.455Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.790220] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9efc25665ffe2330ad130ec","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9efc25665ffe2330ad130ec: start","job_status":"start","queue_duration_s":2.449448,"scheduling_latency_s":2.449469} gitlab # [ 198.796795] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9efc25665ffe2330ad130ec","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9efc25665ffe2330ad130ec: done: 0.0197 sec","job_status":"done","queue_duration_s":2.449448,"scheduling_latency_s":2.449469,"redis_calls":13,"redis_duration_s":0.005431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004495,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0197,"completed_at":"2026-07-06T10:12:05.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.872119] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12e5f4c3e5fef44277c1cd9c","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12e5f4c3e5fef44277c1cd9c: start","job_status":"start","queue_duration_s":2.472721,"scheduling_latency_s":2.472747} gitlab # [ 198.945873] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12e5f4c3e5fef44277c1cd9c","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12e5f4c3e5fef44277c1cd9c: done: 0.026578 sec","job_status":"done","queue_duration_s":2.472721,"scheduling_latency_s":2.472747,"redis_calls":13,"redis_duration_s":0.011774999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026578,"completed_at":"2026-07-06T10:12:05.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.947516] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f77706b6583f24723afc5e6","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f77706b6583f24723afc5e6: start","job_status":"start","queue_duration_s":2.515682,"scheduling_latency_s":2.515706} gitlab # [ 199.032876] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f77706b6583f24723afc5e6","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f77706b6583f24723afc5e6: done: 0.096689 sec","job_status":"done","queue_duration_s":2.515682,"scheduling_latency_s":2.515706,"redis_calls":13,"redis_duration_s":0.07152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.02844,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.04308,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.096689,"completed_at":"2026-07-06T10:12:05.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.035894] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"039eaa33c464dce97e6bcd97","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-039eaa33c464dce97e6bcd97: start","job_status":"start","queue_duration_s":2.617942,"scheduling_latency_s":2.617969} gitlab # [ 199.054774] systemd[1]: Starting nixbot CI service... gitlab # [ 199.056115] gitaly[1385]: time="2026-07-06T10:12:05.347Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:05.346" grpc.time_ms=0.426 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.131103] gitaly[1385]: time="2026-07-06T10:12:06.358Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:06.357" grpc.time_ms=1.59 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.132349] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"039eaa33c464dce97e6bcd97","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-039eaa33c464dce97e6bcd97: done: 0.058485 sec","job_status":"done","queue_duration_s":2.617942,"scheduling_latency_s":2.617969,"redis_calls":22,"redis_duration_s":0.013798,"redis_read_bytes":654,"redis_write_bytes":2027,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.010258,"redis_feature_flag_read_bytes":644,"redis_feature_flag_write_bytes":870,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002559,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.058485,"completed_at":"2026-07-06T10:12:05.683Z","load_balancing_strategy":"primary","db_duration_s":0.017701,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.153152] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80b9021f728920eeb8fad51c","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80b9021f728920eeb8fad51c: start","job_status":"start","queue_duration_s":2.677859,"scheduling_latency_s":2.677918} gitlab # [ 199.233087] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80b9021f728920eeb8fad51c","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80b9021f728920eeb8fad51c: done: 0.079917 sec","job_status":"done","queue_duration_s":2.677859,"scheduling_latency_s":2.677918,"redis_calls":13,"redis_duration_s":0.032546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.031452,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.079917,"completed_at":"2026-07-06T10:12:05.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.306062] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f90a4bf145ff1534b5b33a9","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f90a4bf145ff1534b5b33a9: start","job_status":"start","queue_duration_s":2.765598,"scheduling_latency_s":2.765624} gitlab # [ 199.307056] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f90a4bf145ff1534b5b33a9","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f90a4bf145ff1534b5b33a9: done: 0.044103 sec","job_status":"done","queue_duration_s":2.765598,"scheduling_latency_s":2.765624,"redis_calls":16,"redis_duration_s":0.013448,"redis_read_bytes":140,"redis_write_bytes":1499,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001982,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":342,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00272,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008746,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.044103,"completed_at":"2026-07-06T10:12:05.817Z","load_balancing_strategy":"primary","db_duration_s":0.001162,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.382703] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb0e29d5f1138f5d75eb1541","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb0e29d5f1138f5d75eb1541: start","job_status":"start","queue_duration_s":2.813727,"scheduling_latency_s":2.813751} gitlab # [ 199.385089] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb0e29d5f1138f5d75eb1541","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb0e29d5f1138f5d75eb1541: done: 0.036962 sec","job_status":"done","queue_duration_s":2.813727,"scheduling_latency_s":2.813751,"redis_calls":13,"redis_duration_s":0.008159,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036962,"completed_at":"2026-07-06T10:12:05.858Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.459815] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d093fa96d908d5d379ad90b0","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d093fa96d908d5d379ad90b0: start","job_status":"start","queue_duration_s":2.853118,"scheduling_latency_s":2.85315} gitlab # [ 199.461131] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d093fa96d908d5d379ad90b0","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d093fa96d908d5d379ad90b0: done: 0.029479 sec","job_status":"done","queue_duration_s":2.853118,"scheduling_latency_s":2.85315,"redis_calls":13,"redis_duration_s":0.011547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008702,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029479,"completed_at":"2026-07-06T10:12:05.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.536650] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33af9f67ae8a53a9230887d2","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33af9f67ae8a53a9230887d2: start","job_status":"start","queue_duration_s":2.886495,"scheduling_latency_s":2.886518} gitlab # [ 199.537897] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33af9f67ae8a53a9230887d2","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33af9f67ae8a53a9230887d2: done: 0.039165 sec","job_status":"done","queue_duration_s":2.886495,"scheduling_latency_s":2.886518,"redis_calls":13,"redis_duration_s":0.020095000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002769,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.017326,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039165,"completed_at":"2026-07-06T10:12:05.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.613230] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f194e04ea5d9a0a8ca95323","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f194e04ea5d9a0a8ca95323: start","job_status":"start","queue_duration_s":2.933674,"scheduling_latency_s":2.933701} gitlab # [ 199.614340] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f194e04ea5d9a0a8ca95323","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f194e04ea5d9a0a8ca95323: done: 0.038933 sec","job_status":"done","queue_duration_s":2.933674,"scheduling_latency_s":2.933701,"redis_calls":13,"redis_duration_s":0.021001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00292,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.018081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038933,"completed_at":"2026-07-06T10:12:05.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.692001] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:05.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e6c49d768581e055838cff5","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e6c49d768581e055838cff5: start","job_status":"start","queue_duration_s":2.977596,"scheduling_latency_s":2.977623} gitlab # [ 199.692969] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e6c49d768581e055838cff5","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e6c49d768581e055838cff5: done: 0.028725 sec","job_status":"done","queue_duration_s":2.977596,"scheduling_latency_s":2.977623,"redis_calls":13,"redis_duration_s":0.011615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003431,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028725,"completed_at":"2026-07-06T10:12:06.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.774986] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85a8cfb1bcdcdf89f0564afc","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85a8cfb1bcdcdf89f0564afc: start","job_status":"start","queue_duration_s":3.009236,"scheduling_latency_s":3.009258} gitlab # [ 199.776113] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85a8cfb1bcdcdf89f0564afc","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85a8cfb1bcdcdf89f0564afc: done: 0.027204 sec","job_status":"done","queue_duration_s":3.009236,"scheduling_latency_s":3.009258,"redis_calls":13,"redis_duration_s":0.010755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002404,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027204,"completed_at":"2026-07-06T10:12:06.043Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.847941] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a406df4e4af54f6d8dd0def","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a406df4e4af54f6d8dd0def: start","job_status":"start","queue_duration_s":3.039953,"scheduling_latency_s":3.039978} gitlab # [ 199.849211] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a406df4e4af54f6d8dd0def","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a406df4e4af54f6d8dd0def: done: 0.034268 sec","job_status":"done","queue_duration_s":3.039953,"scheduling_latency_s":3.039978,"redis_calls":13,"redis_duration_s":0.011754,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003431,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008323,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034268,"completed_at":"2026-07-06T10:12:06.081Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 199.925105] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a48cf759855166027391a4f5","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a48cf759855166027391a4f5: start","job_status":"start","queue_duration_s":3.079193,"scheduling_latency_s":3.079219} gitlab # [ 199.926599] gitaly[1385]: time="2026-07-06T10:12:06.488Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:12:16.284" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:06.476" grpc.time_ms=11.282 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.932593] nixbot[3042]: {"ts": "2026-07-06T10:12:09.474Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 199.935959] nixbot[3042]: {"ts": "2026-07-06T10:12:09.566Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 199.938980] nixbot[3042]: {"ts": "2026-07-06T10:12:09.579Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 199.941929] nixbot[3042]: {"ts": "2026-07-06T10:12:09.583Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 200.008357] nixbot[3042]: {"ts": "2026-07-06T10:12:09.586Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 200.009367] nixbot[3042]: {"ts": "2026-07-06T10:12:09.590Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 200.010280] nixbot[3042]: {"ts": "2026-07-06T10:12:09.594Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 200.010995] nixbot[3042]: {"ts": "2026-07-06T10:12:09.596Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 200.011845] nixbot[3042]: {"ts": "2026-07-06T10:12:09.601Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 200.012971] nixbot[3042]: {"ts": "2026-07-06T10:12:09.603Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 200.014100] nixbot[3042]: {"ts": "2026-07-06T10:12:09.606Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 200.014889] nixbot[3042]: {"ts": "2026-07-06T10:12:09.611Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 200.016149] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a48cf759855166027391a4f5","created_at":"2026-07-06T10:12:02.827Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-b6197f42ec5b4c6f","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89ec445bcfb9fcc8fb3e230aefc07dff","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.007Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a48cf759855166027391a4f5: done: 0.073483 sec","job_status":"done","queue_duration_s":3.079193,"scheduling_latency_s":3.079219,"redis_calls":13,"redis_duration_s":0.018265,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.015328,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.073483,"completed_at":"2026-07-06T10:12:06.160Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.030265] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.165Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"d1f46239c69f8c09b0563714","created_at":"2026-07-06T10:12:03.276Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.282Z","job_size_bytes":5,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-d1f46239c69f8c09b0563714: start","job_status":"start","queue_duration_s":2.883368,"scheduling_latency_s":2.883396} gitlab # [ 200.101084] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.303Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"d1f46239c69f8c09b0563714","created_at":"2026-07-06T10:12:03.276Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.282Z","job_size_bytes":5,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-d1f46239c69f8c09b0563714: done: 0.138039 sec","job_status":"done","queue_duration_s":2.883368,"scheduling_latency_s":2.883396,"redis_calls":12,"redis_duration_s":0.012652,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.002922,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007371,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002359,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054532,"duration_s":0.138039,"completed_at":"2026-07-06T10:12:06.303Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059442,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 200.181075] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.309Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"10bbe622e2c23a7b687002a9","created_at":"2026-07-06T10:12:03.621Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.626Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-10bbe622e2c23a7b687002a9: start","job_status":"start","queue_duration_s":2.683266,"scheduling_latency_s":2.683293} gitlab # [ 200.182162] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.762Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"10bbe622e2c23a7b687002a9","created_at":"2026-07-06T10:12:03.621Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.626Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-10bbe622e2c23a7b687002a9: done: 0.452592 sec","job_status":"done","queue_duration_s":2.683266,"scheduling_latency_s":2.683293,"redis_calls":11,"redis_duration_s":0.018942,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001873,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000609,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.108,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.359,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.359,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.102,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.111491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001201828,"duration_s":0.452592,"completed_at":"2026-07-06T10:12:06.762Z","load_balancing_strategy":"primary","db_duration_s":0.305081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.260311] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.766Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8097e909c06d025cdf8679d6","created_at":"2026-07-06T10:12:03.629Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.633Z","job_size_bytes":89,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8097e909c06d025cdf8679d6: start","job_status":"start","queue_duration_s":3.133652,"scheduling_latency_s":3.133675} gitlab # [ 200.261815] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.852Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8097e909c06d025cdf8679d6","created_at":"2026-07-06T10:12:03.629Z","trace_propagation_headers":{"sentry-trace":"6060ccafc12b426bbead00ee3a330cc4-8aefb0d0c7624d3a","baggage":"sentry-trace_id=6060ccafc12b426bbead00ee3a330cc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKMA9J8BHQ9PGBFQ28A1T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:03.633Z","job_size_bytes":89,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8097e909c06d025cdf8679d6: done: 0.086096 sec","job_status":"done","queue_duration_s":3.133652,"scheduling_latency_s":3.133675,"redis_calls":11,"redis_duration_s":0.010247,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.002765,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002176,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000541689,"duration_s":0.086096,"completed_at":"2026-07-06T10:12:06.852Z","load_balancing_strategy":"primary","db_duration_s":0.030729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.344356] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.857Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f92e22e679278fb9c3a42058","created_at":"2026-07-06T10:12:04.749Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-53205b9b5d6b4c70","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:04.755Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f92e22e679278fb9c3a42058: start","job_status":"start","queue_duration_s":2.101585,"scheduling_latency_s":2.101606} gitlab # [ 200.345822] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.880Z","message":"No existing merge request to be cleaned up."} gitlab # [ 200.346912] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.921Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f92e22e679278fb9c3a42058","created_at":"2026-07-06T10:12:04.749Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-53205b9b5d6b4c70","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:04.755Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f92e22e679278fb9c3a42058: done: 0.064192 sec","job_status":"done","queue_duration_s":2.101585,"scheduling_latency_s":2.101606,"redis_calls":10,"redis_duration_s":0.017375,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009156,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.008219,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.064192,"completed_at":"2026-07-06T10:12:06.921Z","load_balancing_strategy":"primary","db_duration_s":0.022753,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.424058] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.927Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b6c35113a35e16dd264912bc","created_at":"2026-07-06T10:12:04.749Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-53205b9b5d6b4c70","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:04.755Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b6c35113a35e16dd264912bc: start","job_status":"start","queue_duration_s":2.17228,"scheduling_latency_s":2.172312} gitlab # [ 200.425162] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.968Z","message":"No existing merge request to be cleaned up."} gitlab # [ 200.425909] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.981Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b6c35113a35e16dd264912bc","created_at":"2026-07-06T10:12:04.749Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-53205b9b5d6b4c70","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:04.755Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b6c35113a35e16dd264912bc: done: 0.053964 sec","job_status":"done","queue_duration_s":2.17228,"scheduling_latency_s":2.172312,"redis_calls":10,"redis_duration_s":0.02444,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.011651,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053964,"completed_at":"2026-07-06T10:12:06.981Z","load_balancing_strategy":"primary","db_duration_s":0.004636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.493237] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:06.985Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0e684ee44c41ab9eac52db74","created_at":"2026-07-06T10:12:04.749Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-53205b9b5d6b4c70","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:04.755Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0e684ee44c41ab9eac52db74: start","job_status":"start","queue_duration_s":2.229685,"scheduling_latency_s":2.229712} gitlab # [ 200.494281] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:07.005Z","message":"No existing merge request to be cleaned up."} gitlab # [ 200.495119] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:07.022Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0e684ee44c41ab9eac52db74","created_at":"2026-07-06T10:12:04.749Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-53205b9b5d6b4c70","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:04.755Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0e684ee44c41ab9eac52db74: done: 0.037191 sec","job_status":"done","queue_duration_s":2.229685,"scheduling_latency_s":2.229712,"redis_calls":10,"redis_duration_s":0.013080999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003422,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.009659,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037191,"completed_at":"2026-07-06T10:12:07.022Z","load_balancing_strategy":"primary","db_duration_s":0.008512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.563288] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:07.023Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d72f06a761f1600a6cf1d45c","created_at":"2026-07-06T10:12:04.749Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-53205b9b5d6b4c70","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:04.755Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d72f06a761f1600a6cf1d45c: start","job_status":"start","queue_duration_s":2.268337,"scheduling_latency_s":2.268364} gitlab # [ 200.631324] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:07.063Z","message":"No existing merge request to be cleaned up."} gitlab # [ 200.632325] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:07.088Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d72f06a761f1600a6cf1d45c","created_at":"2026-07-06T10:12:04.749Z","trace_propagation_headers":{"sentry-trace":"4a2573dc213046db9b9fbdd0c49144fc-53205b9b5d6b4c70","baggage":"sentry-trace_id=4a2573dc213046db9b9fbdd0c49144fc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9c0060849c9b72dd90e0e7ad46a58336","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:04.755Z","job_size_bytes":2,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d72f06a761f1600a6cf1d45c: done: 0.06413 sec","job_status":"done","queue_duration_s":2.268337,"scheduling_latency_s":2.268364,"redis_calls":10,"redis_duration_s":0.016313,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011403,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00491,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06413,"completed_at":"2026-07-06T10:12:07.088Z","load_balancing_strategy":"primary","db_duration_s":0.010516,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.633962] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:07.090Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 0483f69161cd35271d3f18afb21f50f859b3b136 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"35f882921d3cb5347d6ec4c1","created_at":"2026-07-06T10:12:06.462Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-b3cda7fee2154d25","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:06.466Z","job_size_bytes":185,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-35f882921d3cb5347d6ec4c1: start","job_status":"start","queue_duration_s":0.624313,"scheduling_latency_s":0.624336} gitlab # [ 200.711159] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:07.130Z","message":"changes: \"0000000000000000000000000000000000000000 0483f69161cd35271d3f18afb21f50f859b3b136 refs/heads/master\\n\""} gitlab # [ 200.712212] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:08.174Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 0483f69161cd35271d3f18afb21f50f859b3b136 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"35f882921d3cb5347d6ec4c1","created_at":"2026-07-06T10:12:06.462Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-b3cda7fee2154d25","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:06.466Z","job_size_bytes":185,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-35f882921d3cb5347d6ec4c1: done: 1.084323 sec","job_status":"done","queue_duration_s":0.624313,"scheduling_latency_s":0.624336,"gitaly_calls":13,"gitaly_duration_s":0.328338,"redis_calls":199,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.216996,"redis_read_bytes":957,"redis_write_bytes":19725,"redis_cache_calls":14,"redis_cache_duration_s":0.007637,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.029136,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.00401,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4931,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.105811,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.060839,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.009563,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.195,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.224,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.087,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.376568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001354921,"exclusive_lock_hold_duration_s":0.105317854,"duration_s":1.084323,"completed_at":"2026-07-06T10:12:08.174Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.137438,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 200.792793] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:08.176Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c680247ef23f6329fc64cc52","created_at":"2026-07-06T10:12:07.289Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-7237fdf7846d4855","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:07.414Z","job_size_bytes":46,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c680247ef23f6329fc64cc52: start","job_status":"start","queue_duration_s":0.762216,"scheduling_latency_s":0.762238} gitlab # [ 200.794116] gitaly[1385]: time="2026-07-06T10:12:06.510Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1569 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-06T10:12:16.004" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:06.494" grpc.time_ms=16.017 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 200.794833] gitaly[1385]: time="2026-07-06T10:12:06.522Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB duration_ms=92ns method=POST pid=1385 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 200.795951] nixbot[3042]: {"ts": "2026-07-06T10:12:09.614Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 200.796983] nixbot[3042]: {"ts": "2026-07-06T10:12:09.616Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 200.798240] nixbot[3042]: {"ts": "2026-07-06T10:12:09.619Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 200.798922] nixbot[3042]: {"ts": "2026-07-06T10:12:09.621Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 200.800065] nixbot[3042]: {"ts": "2026-07-06T10:12:09.623Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 200.800885] nixbot[3042]: {"ts": "2026-07-06T10:12:09.627Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 200.867317] nixbot[3042]: {"ts": "2026-07-06T10:12:09.631Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 200.868310] nixbot[3042]: {"ts": "2026-07-06T10:12:09.633Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 200.869137] nixbot[3042]: {"ts": "2026-07-06T10:12:09.635Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 200.870079] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:08.249Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c680247ef23f6329fc64cc52","created_at":"2026-07-06T10:12:07.289Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-7237fdf7846d4855","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:07.414Z","job_size_bytes":46,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c680247ef23f6329fc64cc52: done: 0.073424 sec","job_status":"done","queue_duration_s":0.762216,"scheduling_latency_s":0.762238,"gitaly_calls":2,"gitaly_duration_s":0.014274,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0072239999999999995,"redis_read_bytes":68,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001451,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000596,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001615,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00291,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000374908,"duration_s":0.073424,"completed_at":"2026-07-06T10:12:08.249Z","load_balancing_strategy":"primary","db_duration_s":0.010217,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 200.886426] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:08.251Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8d7fa06c4e0eb82f15d67e07","created_at":"2026-07-06T10:12:07.429Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-7237fdf7846d4855","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:07.438Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8d7fa06c4e0eb82f15d67e07: start","job_status":"start","queue_duration_s":0.812979,"scheduling_latency_s":0.813} gitlab # [ 200.964905] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:08.328Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8d7fa06c4e0eb82f15d67e07","created_at":"2026-07-06T10:12:07.429Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-7237fdf7846d4855","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:07.438Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8d7fa06c4e0eb82f15d67e07: done: 0.07693 sec","job_status":"done","queue_duration_s":0.812979,"scheduling_latency_s":0.813,"gitaly_calls":1,"gitaly_duration_s":0.029819,"redis_calls":15,"redis_duration_s":0.0034490000000000002,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000613,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.017,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.057674369,"duration_s":0.07693,"completed_at":"2026-07-06T10:12:08.328Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 200.983701] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:08.329Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"c4baaa32461184188fd888d0","created_at":"2026-07-06T10:12:07.612Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-7237fdf7846d4855","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:07.654Z","job_size_bytes":23,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c4baaa32461184188fd888d0: start","job_status":"start","queue_duration_s":0.675717,"scheduling_latency_s":0.675738} gitlab # [ 201.070381] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:08.366Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"c4baaa32461184188fd888d0","created_at":"2026-07-06T10:12:07.612Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-7237fdf7846d4855","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:07.654Z","job_size_bytes":23,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c4baaa32461184188fd888d0: done: 0.036417 sec","job_status":"done","queue_duration_s":0.675717,"scheduling_latency_s":0.675738,"gitaly_calls":1,"gitaly_duration_s":0.008939,"redis_calls":16,"redis_duration_s":0.003945,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000313,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001236,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001693,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000703,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036417,"completed_at":"2026-07-06T10:12:08.366Z","load_balancing_strategy":"primary","db_duration_s":0.002082,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 201.076107] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:08.367Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"caa85f5c1a19a0714b1eb565","created_at":"2026-07-06T10:12:08.119Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-7237fdf7846d4855","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8a0042ad36f4b014b5f048c96e00a85155bc492b0e27d5ff80a4de1b85bdf29f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:08.123Z","job_size_bytes":467,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-caa85f5c1a19a0714b1eb565: start","job_status":"start","queue_duration_s":0.244111,"scheduling_latency_s":0.244132} gitlab # [ 201.151338] sidekiq-cluster[2108]: {"severity":"WARN","time":"2026-07-06T10:12:09.320Z","class":"CreatePipelineWorker","correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"0483f69161cd35271d3f18afb21f50f859b3b136","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"0483f69161cd35271d3f18afb21f50f859b3b136","ref":"refs/heads/master"},"retry":0} gitlab # [ 201.152343] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:09.326Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"caa85f5c1a19a0714b1eb565","created_at":"2026-07-06T10:12:08.119Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-7237fdf7846d4855","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8a0042ad36f4b014b5f048c96e00a85155bc492b0e27d5ff80a4de1b85bdf29f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:08.123Z","job_size_bytes":467,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-caa85f5c1a19a0714b1eb565: done: 0.958556 sec","job_status":"done","queue_duration_s":0.244111,"scheduling_latency_s":0.244132,"gitaly_calls":7,"gitaly_duration_s":0.065954,"redis_calls":53,"redis_duration_s":0.011779,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.000943,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.005481,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002575,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00114,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.145,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.14,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.649817,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000407593,"duration_s":0.958556,"completed_at":"2026-07-06T10:12:09.326Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.16994,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 201.241141] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:09.328Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"e4ccd0ab81bcaedee168bc03","created_at":"2026-07-06T10:12:08.236Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-cbd147c98aca45cc","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:08.240Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-e4ccd0ab81bcaedee168bc03: start","job_status":"start","queue_duration_s":1.087673,"scheduling_latency_s":1.087746} gitlab # [ 201.242213] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:09.378Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"e4ccd0ab81bcaedee168bc03","created_at":"2026-07-06T10:12:08.236Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-cbd147c98aca45cc","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:08.240Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-e4ccd0ab81bcaedee168bc03: done: 0.050152 sec","job_status":"done","queue_duration_s":1.087673,"scheduling_latency_s":1.087746,"redis_calls":19,"redis_duration_s":0.006128,"redis_read_bytes":21,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001034,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000619,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003581,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010295722,"duration_s":0.050152,"completed_at":"2026-07-06T10:12:09.378Z","load_balancing_strategy":"primary","db_duration_s":0.012109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.322422] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:09.380Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4a568a31161178f9b0ed6a4e","created_at":"2026-07-06T10:12:09.363Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-934bc8fe4cd946c6","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:09.371Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4a568a31161178f9b0ed6a4e: start","job_status":"start","queue_duration_s":0.008764,"scheduling_latency_s":0.008785} gitlab # [ 201.323681] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:09.457Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4a568a31161178f9b0ed6a4e","created_at":"2026-07-06T10:12:09.363Z","trace_propagation_headers":{"sentry-trace":"dfaca7620f8340d09eff7fe846902b19-934bc8fe4cd946c6","baggage":"sentry-trace_id=dfaca7620f8340d09eff7fe846902b19,sentry-release=Unknown"},"correlation_id":"01KWVEKQWT2CC6VERFK4CTZTMB","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:09.371Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4a568a31161178f9b0ed6a4e: done: 0.077016 sec","job_status":"done","queue_duration_s":0.008764,"scheduling_latency_s":0.008785,"redis_calls":11,"redis_duration_s":0.004649,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003764,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000390274,"duration_s":0.077016,"completed_at":"2026-07-06T10:12:09.456Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022203,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 201.394556] gitaly[1385]: time="2026-07-06T10:12:06.522Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4815 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:06.429" grpc.time_ms=93.48 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 201.395869] gitaly[1385]: time="2026-07-06T10:12:06.547Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB diskcache=92037dab-ee59-411d-8825-188a4ccbec2d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:04.726" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 201.406246] gitaly[1385]: time="2026-07-06T10:12:06.547Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=173 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13136 command.oublock=32 command.real_time_ms=1807 command.spawn_token_fork_ms=0 command.system_time_ms=118 command.user_time_ms=55 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:04.726" grpc.time_ms=1821.092 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 201.475810] gitaly[1385]: time="2026-07-06T10:12:07.211Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:17.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:07.201" grpc.time_ms=10.273 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.477116] gitaly[1385]: time="2026-07-06T10:12:07.281Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:12:17.069" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:07.269" grpc.time_ms=11.231 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.477905] gitaly[1385]: time="2026-07-06T10:12:07.474Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=18 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-06T10:12:17.245" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:07.446" grpc.time_ms=28.295 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.487285] gitaly[1385]: time="2026-07-06T10:12:07.519Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3941 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:07.518" grpc.time_ms=0.386 pid=1385 span.kind=server system=grpc gitlab # [ 201.502924] gitaly[1385]: time="2026-07-06T10:12:07.541Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3943 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:07.538" grpc.time_ms=2.474 pid=1385 span.kind=server system=grpc gitlab # [ 201.517906] gitaly[1385]: time="2026-07-06T10:12:07.554Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB diskcache=d85fda92-263c-4b98-bdbe-205d1a1aa5e0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:12:17.290" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:07.490" pid=1385 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 201.533744] gitaly[1385]: time="2026-07-06T10:12:07.555Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=43 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4661 command.oublock=8 command.real_time_ms=58 command.spawn_token_fork_ms=2 command.system_time_ms=25 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:12:17.290" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:07.490" grpc.time_ms=64.02 pid=1385 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 201.600519] gitaly[1385]: time="2026-07-06T10:12:07.659Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T10:12:17.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:07.658" grpc.time_ms=0.155 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.601757] gitaly[1385]: time="2026-07-06T10:12:07.701Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=945 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-06T10:12:17.034" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:07.685" grpc.time_ms=15.812 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.602916] gitaly[1385]: time="2026-07-06T10:12:07.801Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-06T10:12:37.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:07.796" grpc.time_ms=4.961 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.614157] gitaly[1385]: time="2026-07-06T10:12:07.835Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T10:12:17.184" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:07.834" grpc.time_ms=0.206 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.631080] gitaly[1385]: time="2026-07-06T10:12:07.843Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-06T10:12:17.186" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-06T10:12:07.836" grpc.time_ms=6.141 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.699592] gitaly[1385]: time="2026-07-06T10:12:07.863Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:17.203" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:07.853" grpc.time_ms=9.112 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.701052] gitaly[1385]: time="2026-07-06T10:12:07.873Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:17.216" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:07.866" grpc.time_ms=6.756 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.701915] gitaly[1385]: time="2026-07-06T10:12:07.893Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T10:12:37.183" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:07.883" grpc.time_ms=9.838 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.714175] gitaly[1385]: time="2026-07-06T10:12:08.150Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-06T10:12:18.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:08.144" grpc.time_ms=5.494 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.792286] gitaly[1385]: time="2026-07-06T10:12:08.212Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-06T16:12:08.209" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:08.209" grpc.time_ms=2.218 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.793743] gitaly[1385]: time="2026-07-06T10:12:08.226Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-06T10:12:38.018" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:08.219" grpc.time_ms=7.104 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.794801] gitaly[1385]: time="2026-07-06T10:12:08.298Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB diskcache=7522ccb7-9b29-4d00-9f40-41bb40b21c83 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T16:12:08.270" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:08.270" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.795687] gitaly[1385]: time="2026-07-06T10:12:08.298Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1286 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T16:12:08.270" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:08.270" grpc.time_ms=27.719 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.808809] gitaly[1385]: time="2026-07-06T10:12:08.359Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-06T10:12:38.151" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:08.352" grpc.time_ms=6.871 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.829862] gitaly[1385]: time="2026-07-06T10:12:08.401Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:38.199" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:08.400" grpc.time_ms=1.017 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.900367] gitaly[1385]: time="2026-07-06T10:12:08.412Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T10:12:18.262" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:08.405" grpc.time_ms=7.266 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.901859] gitaly[1385]: time="2026-07-06T10:12:08.487Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:38.285" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:08.486" grpc.time_ms=0.895 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.902741] gitaly[1385]: time="2026-07-06T10:12:08.495Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=203 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-06T10:12:38.292" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:08.492" grpc.time_ms=2.756 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.912196] gitaly[1385]: time="2026-07-06T10:12:08.573Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:18.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:08.563" grpc.time_ms=9.257 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.932477] gitaly[1385]: time="2026-07-06T10:12:09.267Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T10:12:39.056" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:09.257" grpc.time_ms=10.664 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.007968] gitaly[1385]: time="2026-07-06T10:12:09.282Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKQWT2CC6VERFK4CTZTMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-06T10:12:39.071" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:09.272" grpc.time_ms=10.275 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.267983] nixbot[3042]: INFO: Started server process [3042] gitlab # [ 202.269913] nixbot[3042]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 202.279800] nixbot[3042]: INFO: ::ffff:127.0.0.1:44108 - "GET /health HTTP/1.1" 200 OK gitlab # [ 202.286135] systemd[1]: Started nixbot CI service. gitlab # [ 202.288987] systemd[1]: Reached target Multi-User System. gitlab # [ 202.291492] systemd[1]: Startup finished in 1.613s (kernel) + 4.299s (initrd) + 3min 16.376s (userspace) = 3min 22.288s. gitlab # [ 202.532707] gitaly[1385]: time="2026-07-06T10:12:12.465Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEKZ9NAPCPAKXA6F3W6CHG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T10:12:42.044" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:12.445" grpc.time_ms=20.433 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.584632] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:12 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 303 gitlab # [ 202.588441] nixbot[3042]: {"ts": "2026-07-06T10:12:12.518Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 202.613957] nixbot[3042]: {"ts": "2026-07-06T10:12:12.548Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 202.736422] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 108 gitlab # [ 202.739050] nixbot[3042]: {"ts": "2026-07-06T10:12:12.672Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 202.740958] nixbot[3042]: {"ts": "2026-07-06T10:12:12.675Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 202.872300] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 111 gitlab # [ 202.881562] nixbot[3042]: {"ts": "2026-07-06T10:12:12.815Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: (finished: waiting for unit nixbot.service, in 6.21 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 203.055797] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:12 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 148 gitlab # [ 203.059186] nixbot[3042]: {"ts": "2026-07-06T10:12:12.990Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 203.092637] nixbot[3042]: INFO: ::ffff:127.0.0.1:59560 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 203.118211] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 52 gitlab # [ 203.127306] nixbot[3042]: {"ts": "2026-07-06T10:12:13.060Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds) (finished: subtest: project discovered and webhook registered, in 0.08 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 203.176282] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 49 gitlab # [ 203.241938] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 25 gitlab # [ 203.251223] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 114 gitlab # [ 203.253083] nixbot[3042]: {"ts": "2026-07-06T10:12:13.186Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 203.323885] gitaly[1385]: time="2026-07-06T10:12:13.257Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM08J8QNKBBFQYVPVNA2Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-06T10:12:43.048" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:13.250" grpc.time_ms=7.334 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.325580] gitaly[1385]: time="2026-07-06T10:12:13.257Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM07S0QTX32ZY2XJ5HDYC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:13.244" grpc.time_ms=13.47 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.360592] gitaly[1385]: time="2026-07-06T10:12:13.294Z" level=info msg="finished unary call with code OK" catfile.duration_ms=19 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=19 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM08J8QNKBBFQYVPVNA2Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:43.074" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:13.275" grpc.time_ms=19.511 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.380823] gitaly[1385]: time="2026-07-06T10:12:13.314Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=14 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM08J8QNKBBFQYVPVNA2Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-06T10:12:23.008" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:13.298" grpc.time_ms=15.862 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.421472] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.54.0" 177 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 203.443639] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 175 gitlab # [ 203.448949] nixbot[3042]: {"ts": "2026-07-06T10:12:13.382Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 203.517936] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 61 gitlab # [ 203.519082] nixbot[3042]: {"ts": "2026-07-06T10:12:13.452Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 203.521612] nixbot[3042]: {"ts": "2026-07-06T10:12:13.455Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 203.610950] gitaly[1385]: time="2026-07-06T10:12:13.544Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0HN2A9RVQPC3NJ5SW6Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:13.537" grpc.time_ms=7.552 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.613318] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 53 gitlab # [ 203.641833] gitaly[1385]: time="2026-07-06T10:12:13.575Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=10 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-06T10:12:23.068" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:13.558" grpc.time_ms=16.645 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.792235] gitaly[1385]: time="2026-07-06T10:12:13.726Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-06T10:12:43.314" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-06T10:12:13.714" grpc.time_ms=11.463 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.856585] gitaly[1385]: time="2026-07-06T10:12:13.790Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q duration_ms=283ns method=POST pid=1385 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 203.889108] gitaly[1385]: time="2026-07-06T10:12:13.823Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q duration_ms=31ns method=POST pid=1385 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 203.890220] gitaly[1385]: time="2026-07-06T10:12:13.823Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5321 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:13.505" grpc.time_ms=318.086 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.935214] gitaly[1385]: time="2026-07-06T10:12:13.868Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0M2J4QNYT0281GF2MQ7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:13.851" pid=1385 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc gitlab # [ 203.936415] gitaly[1385]: time="2026-07-06T10:12:13.868Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0M2J4QNYT0281GF2MQ7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:13.851" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.937421] gitaly[1385]: time="2026-07-06T10:12:13.870Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:13.870" grpc.time_ms=0.442 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.989460] gitlab-workhorse[1287]: time="2026-07-06T10:12:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=149 gitlab # [ 203.990887] gitaly[1385]: time="2026-07-06T10:12:13.923Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0M2J4QNYT0281GF2MQ7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:13.851" grpc.time_ms=18.552 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.015114] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 353 gitlab # [ 204.020090] gitaly[1385]: time="2026-07-06T10:12:13.954Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:13.953" grpc.time_ms=0.453 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 204.057115] gitaly[1385]: time="2026-07-06T10:12:13.991Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:13.990" grpc.time_ms=0.298 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 204.093114] gitaly[1385]: time="2026-07-06T10:12:14.026Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM101EM65X7Z09AS4X5FW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.009" pid=1385 remote_ip="2001:db8:1::1" request_sha=0c733e31ec5a24379b5b0d8b02472ac65b771a6c response_bytes=223 span.kind=server system=grpc gitlab # [ 204.094231] gitaly[1385]: time="2026-07-06T10:12:14.027Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM101EM65X7Z09AS4X5FW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.009" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.095002] gitaly[1385]: time="2026-07-06T10:12:14.028Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM101EM65X7Z09AS4X5FW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.009" grpc.time_ms=18.93 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.095858] gitlab-workhorse[1287]: time="2026-07-06T10:12:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 204.096986] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.54.0" 75 gitlab # [ 204.135988] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:14.069Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0483f69161cd35271d3f18afb21f50f859b3b136 742a8404953aeb6f69baa0955481a1993fe6f49d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cad9cb8e1138374c928f8f18","created_at":"2026-07-06T10:12:14.066Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-5e2b9e3868684b4f","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.067Z","job_size_bytes":185,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cad9cb8e1138374c928f8f18: start","job_status":"start","queue_duration_s":0.001484,"scheduling_latency_s":0.00152} gitlab # [ 204.150842] gitaly[1385]: time="2026-07-06T10:12:14.084Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q duration_ms=45ns method=POST pid=1385 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 204.151885] gitaly[1385]: time="2026-07-06T10:12:14.084Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4815 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:14.038" grpc.time_ms=46.345 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 204.206063] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:14.084Z","message":"changes: \"0483f69161cd35271d3f18afb21f50f859b3b136 742a8404953aeb6f69baa0955481a1993fe6f49d refs/heads/master\\n\""} gitlab # [ 204.219990] gitaly[1385]: time="2026-07-06T10:12:14.154Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q diskcache=403a7f2a-8957-450d-a4d4-613399693994 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:13.444" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 204.235588] gitaly[1385]: time="2026-07-06T10:12:14.154Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=282 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13015 command.oublock=32 command.real_time_ms=696 command.spawn_token_fork_ms=3 command.system_time_ms=218 command.user_time_ms=64 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:13.444" grpc.time_ms=710.155 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # 0483f69..742a840 [ 204.257315] gitlab-workhorse[1287]: time="2026-07-06T10:12:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # master -> master gitlab # [ 204.261547] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:14 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 773 gitlab # [ 204.264856] gitaly[1385]: time="2026-07-06T10:12:14.183Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:24.037" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:14.177" grpc.time_ms=6.166 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.11 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d' gitlab # [ 204.286200] gitaly[1385]: time="2026-07-06T10:12:14.194Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T10:12:24.048" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:14.188" grpc.time_ms=6.061 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.287477] gitaly[1385]: time="2026-07-06T10:12:14.215Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-06T10:12:24.068" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:14.209" grpc.time_ms=6.612 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d', in 0.03 seconds) gitlab # [ 204.299500] gitaly[1385]: time="2026-07-06T10:12:14.233Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=657 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-06T10:12:44.024" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:14.224" grpc.time_ms=7.787 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.313246] gitaly[1385]: time="2026-07-06T10:12:14.247Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-06T10:12:24.103" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-06T10:12:14.243" grpc.time_ms=4.063 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.326353] nixbot[3042]: INFO: ::ffff:127.0.0.1:59570 - "GET /api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d HTTP/1.1" 200 OK gitlab # [ 204.331628] gitaly[1385]: time="2026-07-06T10:12:14.265Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:24.115" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:14.255" grpc.time_ms=10.175 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.451216] gitaly[1385]: time="2026-07-06T10:12:14.385Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM1C3NR7DPRX61EW2EPSB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.374" grpc.time_ms=9.967 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.452675] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 46 gitlab # [ 204.468343] gitaly[1385]: time="2026-07-06T10:12:14.400Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-06T10:12:24.257" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:14.398" grpc.time_ms=2.508 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.483105] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:14.416Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0483f69161cd35271d3f18afb21f50f859b3b136 742a8404953aeb6f69baa0955481a1993fe6f49d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cad9cb8e1138374c928f8f18","created_at":"2026-07-06T10:12:14.066Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-5e2b9e3868684b4f","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.067Z","job_size_bytes":185,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cad9cb8e1138374c928f8f18: done: 0.347373 sec","job_status":"done","queue_duration_s":0.001484,"scheduling_latency_s":0.00152,"gitaly_calls":7,"gitaly_duration_s":0.063546,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.047323,"redis_read_bytes":350,"redis_write_bytes":13067,"redis_cache_calls":9,"redis_cache_duration_s":0.001964,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000641,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001469,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6767,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003138,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.031291,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.00882,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.072,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.017,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.017,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.072,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.165769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000760153,"exclusive_lock_hold_duration_s":0.03616996,"duration_s":0.347373,"completed_at":"2026-07-06T10:12:14.416Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0782,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 204.504868] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:14.419Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"9dd0b482b33c6a61b0bdd63c","created_at":"2026-07-06T10:12:14.197Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-20b879c6ad644ef8","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.199Z","job_size_bytes":46,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9dd0b482b33c6a61b0bdd63c: start","job_status":"start","queue_duration_s":0.21937,"scheduling_latency_s":0.219395} gitlab # [ 204.529510] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:14.454Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"9dd0b482b33c6a61b0bdd63c","created_at":"2026-07-06T10:12:14.197Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-20b879c6ad644ef8","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.199Z","job_size_bytes":46,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9dd0b482b33c6a61b0bdd63c: done: 0.035694 sec","job_status":"done","queue_duration_s":0.21937,"scheduling_latency_s":0.219395,"redis_calls":9,"redis_duration_s":0.006129,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002811,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000472,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002846,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035694,"completed_at":"2026-07-06T10:12:14.454Z","load_balancing_strategy":"primary","db_duration_s":0.005798,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 204.606897] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:14.456Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ccbe0b4db6d5d3e870dc1f6e","created_at":"2026-07-06T10:12:14.204Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-20b879c6ad644ef8","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.205Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ccbe0b4db6d5d3e870dc1f6e: start","job_status":"start","queue_duration_s":0.250845,"scheduling_latency_s":0.250868} gitlab # [ 204.675227] gitlab-workhorse[1287]: time="2026-07-06T10:12:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 204.677504] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 72 gitlab # [ 204.678732] gitaly[1385]: time="2026-07-06T10:12:14.459Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM1DN90Q44EDX85RE4VEX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.444" pid=1385 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab # [ 204.679837] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:14.515Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ccbe0b4db6d5d3e870dc1f6e","created_at":"2026-07-06T10:12:14.204Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-20b879c6ad644ef8","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.205Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ccbe0b4db6d5d3e870dc1f6e: done: 0.059585 sec","job_status":"done","queue_duration_s":0.250845,"scheduling_latency_s":0.250868,"gitaly_calls":1,"gitaly_duration_s":0.028383,"redis_calls":12,"redis_duration_s":0.003692,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000627,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.039158329,"duration_s":0.059585,"completed_at":"2026-07-06T10:12:14.515Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004947,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.774350] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:14.517Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"c914b1e9818d19d34cb7ffef","created_at":"2026-07-06T10:12:14.302Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-20b879c6ad644ef8","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0982fe59fbb9ba5db912c35aa31fc64d2def961eac13e0b0b94570b2eb3debef","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.303Z","job_size_bytes":467,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-c914b1e9818d19d34cb7ffef: start","job_status":"start","queue_duration_s":0.213646,"scheduling_latency_s":0.213673} gitlab # [ 204.775937] gitaly[1385]: time="2026-07-06T10:12:14.459Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM1DN90Q44EDX85RE4VEX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.444" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.776713] gitlab-workhorse[1287]: time="2026-07-06T10:12:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 204.777744] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.54.0" 164 gitlab # [ 204.778409] gitaly[1385]: time="2026-07-06T10:12:14.460Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM1DN90Q44EDX85RE4VEX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.444" grpc.time_ms=15.121 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.845950] gitaly[1385]: time="2026-07-06T10:12:14.501Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q diskcache=43363dc8-6006-4fd6-be18-440169235769 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T16:12:14.474" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:14.475" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.846985] gitaly[1385]: time="2026-07-06T10:12:14.501Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1646 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T16:12:14.474" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:14.475" grpc.time_ms=25.735 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.848105] gitaly[1385]: time="2026-07-06T10:12:14.540Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:44.338" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:14.539" grpc.time_ms=1.107 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.848892] gitaly[1385]: time="2026-07-06T10:12:14.548Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T10:12:24.002" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:14.542" grpc.time_ms=6.131 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.867333] gitaly[1385]: time="2026-07-06T10:12:14.580Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:44.379" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:14.579" grpc.time_ms=1.005 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.939865] gitaly[1385]: time="2026-07-06T10:12:14.588Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=268 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-06T10:12:44.383" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:14.584" grpc.time_ms=4.554 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.941202] gitaly[1385]: time="2026-07-06T10:12:14.626Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM1G0K6GXYXVJT3ZQWS4S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.595" pid=1385 remote_ip="2001:db8:1::1" request_sha=e0abc88fdeab5413648f1acf53ad2c99f533b360 response_bytes=337 span.kind=server system=grpc gitlab # [ 204.941984] gitaly[1385]: time="2026-07-06T10:12:14.627Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1194 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM1G0K6GXYXVJT3ZQWS4S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.595" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.947327] gitaly[1385]: time="2026-07-06T10:12:14.627Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1194 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM1G0K6GXYXVJT3ZQWS4S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:14.595" grpc.time_ms=31.433 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.970487] gitaly[1385]: time="2026-07-06T10:12:14.637Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:24.070" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:14.630" grpc.time_ms=6.282 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.125835] gitaly[1385]: time="2026-07-06T10:12:15.059Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T10:12:45.048" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:15.048" grpc.time_ms=11.258 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.141614] gitaly[1385]: time="2026-07-06T10:12:15.075Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM21A1NND86JJWC8WD859 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.068" grpc.time_ms=7.106 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.145650] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 59 gitlab # [ 205.186337] gitaly[1385]: time="2026-07-06T10:12:15.120Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM0E64ZT9PCJ6C8C67H6Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-06T10:12:45.109" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:15.110" grpc.time_ms=10.189 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.196907] gitaly[1385]: time="2026-07-06T10:12:15.130Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2382Q581WQNT073G90P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.118" pid=1385 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=214 span.kind=server system=grpc gitlab # [ 205.197723] gitaly[1385]: time="2026-07-06T10:12:15.131Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2382Q581WQNT073G90P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.118" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.198837] gitaly[1385]: time="2026-07-06T10:12:15.131Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2382Q581WQNT073G90P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.118" grpc.time_ms=13.17 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.201540] gitlab-workhorse[1287]: time="2026-07-06T10:12:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=214 gitlab # [ 205.202359] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 214 "" "git/2.54.0" 51 gitlab # [ 205.203578] sidekiq-cluster[2108]: {"severity":"WARN","time":"2026-07-06T10:12:15.134Z","class":"CreatePipelineWorker","correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0483f69161cd35271d3f18afb21f50f859b3b136","after":"742a8404953aeb6f69baa0955481a1993fe6f49d","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"742a8404953aeb6f69baa0955481a1993fe6f49d","ref":"refs/heads/master"},"retry":0} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d' gitlab # [ 205.269931] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:15.141Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"c914b1e9818d19d34cb7ffef","created_at":"2026-07-06T10:12:14.302Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-20b879c6ad644ef8","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0982fe59fbb9ba5db912c35aa31fc64d2def961eac13e0b0b94570b2eb3debef","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.303Z","job_size_bytes":467,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-c914b1e9818d19d34cb7ffef: done: 0.623852 sec","job_status":"done","queue_duration_s":0.213646,"scheduling_latency_s":0.213673,"gitaly_calls":7,"gitaly_duration_s":0.110635,"redis_calls":33,"redis_duration_s":0.009151,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000707,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003364,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003699,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.477582,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.623852,"completed_at":"2026-07-06T10:12:15.141Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022652,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 205.287853] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:15.143Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"3ec077b4-f47a-40bb-8554-221e6e5de6b0\"}"],"class":"WebHookWorker","jid":"de66940d1139cf0975847557","created_at":"2026-07-06T10:12:14.383Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-20b879c6ad644ef8","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:826075d8b29c18390f0a6f89caecdc4b26f37f9cc4c7a7e2060c7847d3773c41","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.387Z","job_size_bytes":1749,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-de66940d1139cf0975847557: start","job_status":"start","queue_duration_s":0.755172,"scheduling_latency_s":0.7552} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d', in 0.07 seconds) gitlab # [ 205.354531] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:15.221Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"3ec077b4-f47a-40bb-8554-221e6e5de6b0\"}"],"class":"WebHookWorker","jid":"de66940d1139cf0975847557","created_at":"2026-07-06T10:12:14.383Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-20b879c6ad644ef8","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:826075d8b29c18390f0a6f89caecdc4b26f37f9cc4c7a7e2060c7847d3773c41","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:14.387Z","job_size_bytes":1749,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-de66940d1139cf0975847557: done: 0.078703 sec","job_status":"done","queue_duration_s":0.755172,"scheduling_latency_s":0.7552,"redis_calls":19,"redis_duration_s":0.004363,"redis_read_bytes":31,"redis_write_bytes":5283,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001031,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000459,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3645,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002016,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000857,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.00892767100000924,"cpu_s":0.055877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352559,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.078703,"completed_at":"2026-07-06T10:12:15.221Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 205.433329] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:15.223Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","de66940d1139cf0975847557"],"class":"WebHooks::LogExecutionWorker","jid":"19773340062a023d69439ab2","created_at":"2026-07-06T10:12:15.207Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-576959804020406d","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:55e42ac7d2dc85d62d9a693d654fcff85348e17f07994845272deb908335ce2f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:15.215Z","job_size_bytes":2508,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-19773340062a023d69439ab2: start","job_status":"start","queue_duration_s":0.00802,"scheduling_latency_s":0.008042} gitlab # [ 205.437684] nixbot[3042]: INFO: ::ffff:127.0.0.1:59574 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 205.438810] nixbot[3042]: INFO: ::ffff:127.0.0.1:59578 - "GET /api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d HTTP/1.1" 200 OK gitlab # [ 205.441920] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 88 gitlab # [ 205.443057] gitaly[1385]: time="2026-07-06T10:12:15.315Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM27W9AH551NAHFW8EXHX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.306" grpc.time_ms=9.679 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.443922] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:15.346Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","de66940d1139cf0975847557"],"class":"WebHooks::LogExecutionWorker","jid":"19773340062a023d69439ab2","created_at":"2026-07-06T10:12:15.207Z","trace_propagation_headers":{"sentry-trace":"28b283e24c834bf2bd33448e5fa3906f-576959804020406d","baggage":"sentry-trace_id=28b283e24c834bf2bd33448e5fa3906f,sentry-release=Unknown"},"correlation_id":"01KWVEM0E64ZT9PCJ6C8C67H6Q","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:55e42ac7d2dc85d62d9a693d654fcff85348e17f07994845272deb908335ce2f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:15.215Z","job_size_bytes":2508,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-19773340062a023d69439ab2: done: 0.123306 sec","job_status":"done","queue_duration_s":0.00802,"scheduling_latency_s":0.008042,"redis_calls":10,"redis_duration_s":0.006109,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003064,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002585,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.025,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000671314,"duration_s":0.123306,"completed_at":"2026-07-06T10:12:15.346Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01113,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 205.522470] gitaly[1385]: time="2026-07-06T10:12:15.397Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2ASTKVHYWPY540NG9YF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.386" pid=1385 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab # [ 205.526666] gitlab-workhorse[1287]: time="2026-07-06T10:12:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 205.529950] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 77 gitlab # [ 205.532806] gitaly[1385]: time="2026-07-06T10:12:15.397Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2ASTKVHYWPY540NG9YF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.386" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.599529] gitaly[1385]: time="2026-07-06T10:12:15.398Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2ASTKVHYWPY540NG9YF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.386" grpc.time_ms=11.737 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.600991] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 65 gitlab # [ 205.602372] gitaly[1385]: time="2026-07-06T10:12:15.491Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2E3F562EXTPB8SF3VTN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.483" grpc.time_ms=8.059 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.682099] gitaly[1385]: time="2026-07-06T10:12:15.614Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2G64XG25966F5HD70YY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.598" pid=1385 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 205.683335] gitaly[1385]: time="2026-07-06T10:12:15.614Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1177 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2G64XG25966F5HD70YY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.598" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.685084] gitlab-workhorse[1287]: time="2026-07-06T10:12:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 205.685867] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.54.0" 120 gitlab # [ 205.687002] gitaly[1385]: time="2026-07-06T10:12:15.614Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1177 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2G64XG25966F5HD70YY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:15.598" grpc.time_ms=16.202 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 205.778235] nixbot[3042]: {"ts": "2026-07-06T10:12:15.712Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"} gitlab # [ 205.824860] gitaly[1385]: time="2026-07-06T10:12:15.759Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:45.356" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:15.757" grpc.time_ms=1.57 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.838193] gitaly[1385]: time="2026-07-06T10:12:15.772Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:45.570" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:15.771" grpc.time_ms=0.635 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.873133] gitaly[1385]: time="2026-07-06T10:12:15.807Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-06T10:12:45.598" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:15.799" grpc.time_ms=7.791 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.887101] gitaly[1385]: time="2026-07-06T10:12:15.821Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:25.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:15.815" grpc.time_ms=5.568 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.931351] gitaly[1385]: time="2026-07-06T10:12:15.865Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-06T10:12:45.458" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:15.858" grpc.time_ms=6.759 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.940617] gitaly[1385]: time="2026-07-06T10:12:15.874Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T10:12:25.064" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:15.874" grpc.time_ms=0.216 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.953614] gitaly[1385]: time="2026-07-06T10:12:15.887Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T10:12:25.071" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:15.882" grpc.time_ms=5.579 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.985528] gitaly[1385]: time="2026-07-06T10:12:15.919Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:15.918" grpc.time_ms=0.696 pid=1385 span.kind=server system=grpc gitlab # [ 206.009113] gitaly[1385]: time="2026-07-06T10:12:15.943Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4160 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:15.942" grpc.time_ms=0.298 pid=1385 span.kind=server system=grpc gitlab # [ 206.031348] gitaly[1385]: time="2026-07-06T10:12:15.965Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y diskcache=30da9579-4d88-47ba-a02a-8c66ec491eef grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:12:25.079" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:15.889" pid=1385 ref=refs/keep-around/742a8404953aeb6f69baa0955481a1993fe6f49d remote_ip="2001:db8:1::1" revision=742a8404953aeb6f69baa0955481a1993fe6f49d span.kind=server system=grpc username=root gitlab # [ 206.032554] gitaly[1385]: time="2026-07-06T10:12:15.965Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=67 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5242 command.oublock=8 command.real_time_ms=68 command.spawn_token_fork_ms=0 command.system_time_ms=44 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:12:25.079" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:15.889" grpc.time_ms=75.82 pid=1385 ref=refs/keep-around/742a8404953aeb6f69baa0955481a1993fe6f49d remote_ip="2001:db8:1::1" revision=742a8404953aeb6f69baa0955481a1993fe6f49d span.kind=server system=grpc username=root gitlab # [ 206.036473] gitaly[1385]: time="2026-07-06T10:12:15.970Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:45.769" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:15.970" grpc.time_ms=0.425 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.041131] gitaly[1385]: time="2026-07-06T10:12:15.971Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:25.053" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:15.963" grpc.time_ms=7.567 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.058769] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:15.992Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1d3eafb36041c435c071735b","created_at":"2026-07-06T10:12:15.982Z","trace_propagation_headers":{"sentry-trace":"8595ca7207f64a469aa357f7726df9e9-a1912533b39a4a57","baggage":"sentry-trace_id=8595ca7207f64a469aa357f7726df9e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2Q562VPCTWXDG6S108Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:15.990Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1d3eafb36041c435c071735b: start","job_status":"start","queue_duration_s":0.001979,"scheduling_latency_s":0.002029} gitlab # [ 206.089358] gitaly[1385]: time="2026-07-06T10:12:16.023Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T10:12:26.002" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:16.023" grpc.time_ms=0.217 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.103392] gitaly[1385]: time="2026-07-06T10:12:16.037Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T10:12:26.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:16.029" grpc.time_ms=8.262 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.126721] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.060Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1d3eafb36041c435c071735b","created_at":"2026-07-06T10:12:15.982Z","trace_propagation_headers":{"sentry-trace":"8595ca7207f64a469aa357f7726df9e9-a1912533b39a4a57","baggage":"sentry-trace_id=8595ca7207f64a469aa357f7726df9e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2Q562VPCTWXDG6S108Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:15.990Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1d3eafb36041c435c071735b: done: 0.068274 sec","job_status":"done","queue_duration_s":0.001979,"scheduling_latency_s":0.002029,"redis_calls":19,"redis_duration_s":0.009124,"redis_read_bytes":145,"redis_write_bytes":2300,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.007056,"redis_feature_flag_read_bytes":138,"redis_feature_flag_write_bytes":1482,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000618,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345574,"duration_s":0.068274,"completed_at":"2026-07-06T10:12:16.060Z","load_balancing_strategy":"primary","db_duration_s":0.012466,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 206.220784] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.063Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"61a7b2095bd062f86ac23de7","created_at":"2026-07-06T10:12:15.991Z","trace_propagation_headers":{"sentry-trace":"8595ca7207f64a469aa357f7726df9e9-a1912533b39a4a57","baggage":"sentry-trace_id=8595ca7207f64a469aa357f7726df9e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2Q562VPCTWXDG6S108Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:15.998Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-61a7b2095bd062f86ac23de7: start","job_status":"start","queue_duration_s":0.065011,"scheduling_latency_s":0.065034} gitlab # [ 206.221967] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.122Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"61a7b2095bd062f86ac23de7","created_at":"2026-07-06T10:12:15.991Z","trace_propagation_headers":{"sentry-trace":"8595ca7207f64a469aa357f7726df9e9-a1912533b39a4a57","baggage":"sentry-trace_id=8595ca7207f64a469aa357f7726df9e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2Q562VPCTWXDG6S108Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:15.998Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-61a7b2095bd062f86ac23de7: done: 0.059327 sec","job_status":"done","queue_duration_s":0.065011,"scheduling_latency_s":0.065034,"redis_calls":15,"redis_duration_s":0.005511,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000839,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002401,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002271,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000573816,"duration_s":0.059327,"completed_at":"2026-07-06T10:12:16.122Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 206.332409] gitaly[1385]: time="2026-07-06T10:12:16.118Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:16.116" grpc.time_ms=2.331 pid=1385 span.kind=server system=grpc gitlab # [ 206.335287] gitaly[1385]: time="2026-07-06T10:12:16.164Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4160 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:16.164" grpc.time_ms=0.425 pid=1385 span.kind=server system=grpc gitlab # [ 206.336740] nixbot[3042]: {"ts": "2026-07-06T10:12:16.150Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/742a8404953aeb6f69baa0955481a1993fe6f49d \"HTTP/1.1 201 Created\""} gitlab # [ 206.337748] nixbot[3042]: {"ts": "2026-07-06T10:12:16.165Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-742a8404953a-6e94165f", "argv0": "bwrap"} gitlab # [ 206.339994] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/742a8404953aeb6f69baa0955481a1993fe6f49d HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 431 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d' gitlab # [ 206.343204] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.244Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5333e7770da77fb19b022c64","created_at":"2026-07-06T10:12:16.117Z","trace_propagation_headers":{"sentry-trace":"8595ca7207f64a469aa357f7726df9e9-a1912533b39a4a57","baggage":"sentry-trace_id=8595ca7207f64a469aa357f7726df9e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2Q562VPCTWXDG6S108Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.127Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5333e7770da77fb19b022c64: start","job_status":"start","queue_duration_s":0.117763,"scheduling_latency_s":0.117815} gitlab # [ 206.344389] gitaly[1385]: time="2026-07-06T10:12:16.210Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 diskcache=cd47f63d-4d21-4f6d-9903-5b20fb7eebfd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:12:26.019" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:16.039" pid=1385 ref=refs/keep-around/0483f69161cd35271d3f18afb21f50f859b3b136 remote_ip="2001:db8:1::1" revision=0483f69161cd35271d3f18afb21f50f859b3b136 span.kind=server system=grpc username=root gitlab # [ 206.351833] gitaly[1385]: time="2026-07-06T10:12:16.210Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5149 command.oublock=8 command.real_time_ms=158 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=22 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T10:12:26.019" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T10:12:16.039" grpc.time_ms=170.562 pid=1385 ref=refs/keep-around/0483f69161cd35271d3f18afb21f50f859b3b136 remote_ip="2001:db8:1::1" revision=0483f69161cd35271d3f18afb21f50f859b3b136 span.kind=server system=grpc username=root gitlab # [ 206.422191] gitaly[1385]: time="2026-07-06T10:12:16.216Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:46.014" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:16.215" grpc.time_ms=0.581 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.423685] gitaly[1385]: time="2026-07-06T10:12:16.328Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:46.126" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:16.327" grpc.time_ms=0.728 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.427866] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0483f69161cd35271d3f18afb21f50f859b3b136 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 660 gitlab # [ 206.431185] gitaly[1385]: time="2026-07-06T10:12:16.365Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:46.163" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:16.364" grpc.time_ms=0.682 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.457480] gitaly[1385]: time="2026-07-06T10:12:16.391Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2Q562VPCTWXDG6S108Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:46.190" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:16.390" grpc.time_ms=0.732 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.467415] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.401Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5333e7770da77fb19b022c64","created_at":"2026-07-06T10:12:16.117Z","trace_propagation_headers":{"sentry-trace":"8595ca7207f64a469aa357f7726df9e9-a1912533b39a4a57","baggage":"sentry-trace_id=8595ca7207f64a469aa357f7726df9e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2Q562VPCTWXDG6S108Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.127Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5333e7770da77fb19b022c64: done: 0.156556 sec","job_status":"done","queue_duration_s":0.117763,"scheduling_latency_s":0.117815,"gitaly_calls":3,"gitaly_duration_s":0.008947,"redis_calls":53,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.01486,"redis_read_bytes":235,"redis_write_bytes":7159,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001041,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002319,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001756,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000318,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001976,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000533,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.004432,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001444,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096061,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.124752727,"duration_s":0.156556,"completed_at":"2026-07-06T10:12:16.401Z","load_balancing_strategy":"primary","db_duration_s":0.037644,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 206.557140] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.408Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8cebbbc2b79f672295785e91","created_at":"2026-07-06T10:12:16.219Z","trace_propagation_headers":{"sentry-trace":"62f023161c4349dcba13cb2685433327-3d573ad303214a39","baggage":"sentry-trace_id=62f023161c4349dcba13cb2685433327,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2PNG77SEY6QWDF7F516","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.225Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8cebbbc2b79f672295785e91: start","job_status":"start","queue_duration_s":0.182166,"scheduling_latency_s":0.182205} gitlab # [ 206.563130] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.431Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8cebbbc2b79f672295785e91","created_at":"2026-07-06T10:12:16.219Z","trace_propagation_headers":{"sentry-trace":"62f023161c4349dcba13cb2685433327-3d573ad303214a39","baggage":"sentry-trace_id=62f023161c4349dcba13cb2685433327,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2PNG77SEY6QWDF7F516","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.225Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8cebbbc2b79f672295785e91: done: 0.023794 sec","job_status":"done","queue_duration_s":0.182166,"scheduling_latency_s":0.182205,"redis_calls":7,"redis_duration_s":0.001697,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023794,"completed_at":"2026-07-06T10:12:16.431Z","load_balancing_strategy":"primary","db_duration_s":0.004076,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 206.647297] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.433Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f6a0d65dcf01f940aa673d62","created_at":"2026-07-06T10:12:16.227Z","trace_propagation_headers":{"sentry-trace":"62f023161c4349dcba13cb2685433327-3d573ad303214a39","baggage":"sentry-trace_id=62f023161c4349dcba13cb2685433327,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2PNG77SEY6QWDF7F516","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.230Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f6a0d65dcf01f940aa673d62: start","job_status":"start","queue_duration_s":0.203035,"scheduling_latency_s":0.203059} gitlab # [ 206.716512] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.472Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f6a0d65dcf01f940aa673d62","created_at":"2026-07-06T10:12:16.227Z","trace_propagation_headers":{"sentry-trace":"62f023161c4349dcba13cb2685433327-3d573ad303214a39","baggage":"sentry-trace_id=62f023161c4349dcba13cb2685433327,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2PNG77SEY6QWDF7F516","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.230Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f6a0d65dcf01f940aa673d62: done: 0.039287 sec","job_status":"done","queue_duration_s":0.203035,"scheduling_latency_s":0.203059,"redis_calls":10,"redis_duration_s":0.002473,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001238,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039287,"completed_at":"2026-07-06T10:12:16.472Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009688,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 206.720386] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.474Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"5da22877ae126ffe7dfe5b50","created_at":"2026-07-06T10:12:16.313Z","trace_propagation_headers":{"sentry-trace":"8595ca7207f64a469aa357f7726df9e9-d10991827a0d4fcb","baggage":"sentry-trace_id=8595ca7207f64a469aa357f7726df9e9,sentry-release=Unknown"},"correlation_id":"01KWVEM2Q562VPCTWXDG6S108Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.317Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5da22877ae126ffe7dfe5b50: start","job_status":"start","queue_duration_s":0.157188,"scheduling_latency_s":0.157214} gitlab # [ 206.791969] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.505Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"5da22877ae126ffe7dfe5b50","created_at":"2026-07-06T10:12:16.313Z","trace_propagation_headers":{"sentry-trace":"8595ca7207f64a469aa357f7726df9e9-d10991827a0d4fcb","baggage":"sentry-trace_id=8595ca7207f64a469aa357f7726df9e9,sentry-release=Unknown"},"correlation_id":"01KWVEM2Q562VPCTWXDG6S108Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.317Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5da22877ae126ffe7dfe5b50: done: 0.031466 sec","job_status":"done","queue_duration_s":0.157188,"scheduling_latency_s":0.157214,"redis_calls":10,"redis_duration_s":0.002477,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001534,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000338032,"duration_s":0.031466,"completed_at":"2026-07-06T10:12:16.505Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 206.814451] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.507Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c838e36ad3096fa9cbe7cbe6","created_at":"2026-07-06T10:12:16.336Z","trace_propagation_headers":{"sentry-trace":"62f023161c4349dcba13cb2685433327-3d573ad303214a39","baggage":"sentry-trace_id=62f023161c4349dcba13cb2685433327,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2PNG77SEY6QWDF7F516","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.340Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c838e36ad3096fa9cbe7cbe6: start","job_status":"start","queue_duration_s":0.166597,"scheduling_latency_s":0.166617} gitlab # [ 206.839336] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.703Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c838e36ad3096fa9cbe7cbe6","created_at":"2026-07-06T10:12:16.336Z","trace_propagation_headers":{"sentry-trace":"62f023161c4349dcba13cb2685433327-3d573ad303214a39","baggage":"sentry-trace_id=62f023161c4349dcba13cb2685433327,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM2PNG77SEY6QWDF7F516","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.340Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c838e36ad3096fa9cbe7cbe6: done: 0.196491 sec","job_status":"done","queue_duration_s":0.166597,"scheduling_latency_s":0.166617,"gitaly_calls":3,"gitaly_duration_s":0.009742,"redis_calls":39,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.00705,"redis_read_bytes":71,"redis_write_bytes":5658,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001041,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001354,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_queues_calls":2,"redis_queues_duration_s":0.000457,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001649,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000231,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000254,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.161983,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.176144175,"duration_s":0.196491,"completed_at":"2026-07-06T10:12:16.703Z","load_balancing_strategy":"primary","db_duration_s":0.022696,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 206.926451] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.705Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"0028af8466585736e6735b73","created_at":"2026-07-06T10:12:16.544Z","trace_propagation_headers":{"sentry-trace":"62f023161c4349dcba13cb2685433327-529f5a0af9af453b","baggage":"sentry-trace_id=62f023161c4349dcba13cb2685433327,sentry-release=Unknown"},"correlation_id":"01KWVEM2PNG77SEY6QWDF7F516","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.546Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-0028af8466585736e6735b73: start","job_status":"start","queue_duration_s":0.159067,"scheduling_latency_s":0.159092} gitlab # [ 206.995543] gitaly[1385]: time="2026-07-06T10:12:16.555Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:46.353" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:16.554" grpc.time_ms=0.984 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.997439] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:16.733Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"0028af8466585736e6735b73","created_at":"2026-07-06T10:12:16.544Z","trace_propagation_headers":{"sentry-trace":"62f023161c4349dcba13cb2685433327-529f5a0af9af453b","baggage":"sentry-trace_id=62f023161c4349dcba13cb2685433327,sentry-release=Unknown"},"correlation_id":"01KWVEM2PNG77SEY6QWDF7F516","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:16.546Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-0028af8466585736e6735b73: done: 0.027821 sec","job_status":"done","queue_duration_s":0.159067,"scheduling_latency_s":0.159092,"redis_calls":7,"redis_duration_s":0.002175,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027821,"completed_at":"2026-07-06T10:12:16.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 207.072569] gitaly[1385]: time="2026-07-06T10:12:16.586Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:46.385" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:16.585" grpc.time_ms=0.613 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.074235] gitaly[1385]: time="2026-07-06T10:12:16.694Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM2PNG77SEY6QWDF7F516 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:46.492" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:16.693" grpc.time_ms=0.701 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.153866] nixbot[3042]: {"ts": "2026-07-06T10:12:17.086Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0483f69161cd35271d3f18afb21f50f859b3b136 \"HTTP/1.1 201 Created\""} gitlab # [ 207.168993] nixbot[3042]: INFO: ::ffff:127.0.0.1:59586 - "GET /api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=742a8404953aeb6f69baa0955481a1993fe6f49d', in 0.79 seconds) (finished: subtest: push delivers the webhook, in 4.00 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 207.242404] systemd[1]: Started Nix Daemon. gitlab # [ 207.245873] gitaly[1385]: time="2026-07-06T10:12:17.180Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:47.078" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:17.179" grpc.time_ms=0.745 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.273122] gitaly[1385]: time="2026-07-06T10:12:17.204Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM43SVA50QQJ178A0M0VK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:47.002" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:17.202" grpc.time_ms=1.72 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.284411] gitaly[1385]: time="2026-07-06T10:12:17.218Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:27.189" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:17.209" grpc.time_ms=8.736 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.295532] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 83 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.14 seconds) gitlab # [ 207.341590] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:17.273Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8877894235d2a49e42a8ba96","created_at":"2026-07-06T10:12:17.268Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-5e8642c7dd68496a","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.271Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8877894235d2a49e42a8ba96: start","job_status":"start","queue_duration_s":0.001823,"scheduling_latency_s":0.001873} gitlab # [ 207.353356] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0483f69161cd35271d3f18afb21f50f859b3b136 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 170 gitlab # [ 207.354364] nixbot[3042]: {"ts": "2026-07-06T10:12:17.288Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0483f69161cd35271d3f18afb21f50f859b3b136 \"HTTP/1.1 201 Created\""} gitlab # [ 207.365540] nix-daemon[3457]: accepted connection from pid 3455, user nixbot gitlab # [ 207.378144] nix-daemon[3457]: accepted connection from pid 3456, user nixbot gitlab # [ 207.399320] gitaly[1385]: time="2026-07-06T10:12:17.333Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM48CSJPTR0CSW3QV1J8G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:47.131" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:17.332" grpc.time_ms=0.499 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.455161] gitaly[1385]: time="2026-07-06T10:12:17.389Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:17.388" grpc.time_ms=0.377 pid=1385 span.kind=server system=grpc gitlab # [ 207.493947] gitaly[1385]: time="2026-07-06T10:12:17.428Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:17.427" grpc.time_ms=0.509 pid=1385 span.kind=server system=grpc gitlab # [ 207.496451] gitaly[1385]: time="2026-07-06T10:12:17.430Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM48CSJPTR0CSW3QV1J8G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:27.134" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:17.424" grpc.time_ms=5.806 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.538417] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0483f69161cd35271d3f18afb21f50f859b3b136 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 179 gitlab # [ 207.544143] nixbot[3042]: {"ts": "2026-07-06T10:12:17.477Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0483f69161cd35271d3f18afb21f50f859b3b136 \"HTTP/1.1 201 Created\""} gitlab # [ 207.557818] gitaly[1385]: time="2026-07-06T10:12:17.492Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:17.491" grpc.time_ms=0.375 pid=1385 span.kind=server system=grpc gitlab # [ 207.581444] gitaly[1385]: time="2026-07-06T10:12:17.515Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS diskcache=d60ae1e5-cabc-4396-b7a6-97a3a53b6d3b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:47.163" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:17.364" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.582756] gitaly[1385]: time="2026-07-06T10:12:17.515Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=138 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7127 command.oublock=0 command.real_time_ms=143 command.spawn_token_fork_ms=0 command.system_time_ms=89 command.user_time_ms=49 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:47.163" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:17.364" grpc.time_ms=151.666 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.713436] gitaly[1385]: time="2026-07-06T10:12:17.647Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:47.444" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:17.645" grpc.time_ms=1.43 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.807350] gitaly[1385]: time="2026-07-06T10:12:17.741Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:47.538" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:17.739" grpc.time_ms=1.377 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.927650] gitaly[1385]: time="2026-07-06T10:12:17.861Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:47.658" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:17.859" grpc.time_ms=1.307 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.992460] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:17.925Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"daef023fb09c3a43e8475a55","created_at":"2026-07-06T10:12:17.920Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"8877894235d2a49e42a8ba96","job_size_bytes":3,"pid":2108,"job_status":"deduplicated","message":"PipelineProcessWorker JID-daef023fb09c3a43e8475a55: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 207.996220] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:17.929Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8877894235d2a49e42a8ba96","created_at":"2026-07-06T10:12:17.268Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-5e8642c7dd68496a","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.271Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2108,"job_status":"rescheduled","message":"PipelineProcessWorker JID-8877894235d2a49e42a8ba96: rescheduled"} gitlab # [ 208.010508] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:17.943Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8877894235d2a49e42a8ba96","created_at":"2026-07-06T10:12:17.268Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-5e8642c7dd68496a","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.271Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8877894235d2a49e42a8ba96: done: 0.670198 sec","job_status":"done","queue_duration_s":0.001823,"scheduling_latency_s":0.001873,"gitaly_calls":4,"gitaly_duration_s":0.293133,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.043779,"redis_read_bytes":264,"redis_write_bytes":24007,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001254,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003165,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.013927,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.006227,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12386,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.01204,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000311,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000702,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004899,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.101,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.013,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.075,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.233939,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.610172242,"duration_s":0.670198,"completed_at":"2026-07-06T10:12:17.943Z","load_balancing_strategy":"primary","db_duration_s":0.106093,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 208.109291] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:17.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"305910fe5345d5cad0d9caf8","created_at":"2026-07-06T10:12:17.351Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.355Z","job_size_bytes":23,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-305910fe5345d5cad0d9caf8: start","job_status":"start","queue_duration_s":0.592422,"scheduling_latency_s":0.592477} gitlab # [ 208.128260] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.061Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"305910fe5345d5cad0d9caf8","created_at":"2026-07-06T10:12:17.351Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.355Z","job_size_bytes":23,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-305910fe5345d5cad0d9caf8: done: 0.114612 sec","job_status":"done","queue_duration_s":0.592422,"scheduling_latency_s":0.592477,"redis_calls":15,"redis_duration_s":0.007722,"redis_read_bytes":13,"redis_write_bytes":3010,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000644,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1625,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003787,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000919,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079054,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000549232,"duration_s":0.114612,"completed_at":"2026-07-06T10:12:18.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021414,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 208.223873] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.065Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8d080782dc965390e7993b5b","created_at":"2026-07-06T10:12:17.557Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.563Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8d080782dc965390e7993b5b: start","job_status":"start","queue_duration_s":0.501428,"scheduling_latency_s":0.50145} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 208.225375] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.094Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8d080782dc965390e7993b5b","created_at":"2026-07-06T10:12:17.557Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.563Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8d080782dc965390e7993b5b: done: 0.029258 sec","job_status":"done","queue_duration_s":0.501428,"scheduling_latency_s":0.50145,"redis_calls":10,"redis_duration_s":0.004647,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001037,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000680534,"duration_s":0.029258,"completed_at":"2026-07-06T10:12:18.094Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006069,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.271912] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.096Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"1d1f3e66524288300030d12a","created_at":"2026-07-06T10:12:18.052Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-e65f69b3b0154303","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:23d53ec14eb3e3f66280e6f0d22e9cbdf3faec137f1ba15faa91139258418662","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:18.055Z","job_size_bytes":464,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-1d1f3e66524288300030d12a: start","job_status":"start","queue_duration_s":0.041388,"scheduling_latency_s":0.041408} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.20 seconds) gitlab # [ 208.339731] sidekiq-cluster[2108]: {"severity":"WARN","time":"2026-07-06T10:12:18.135Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"1d1f3e66524288300030d12a","created_at":"2026-07-06T10:12:18.052Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-e65f69b3b0154303","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:23d53ec14eb3e3f66280e6f0d22e9cbdf3faec137f1ba15faa91139258418662","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:18.055Z","job_size_bytes":464,"pid":2108,"sidekiq_tid":"m318","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-1d1f3e66524288300030d12a: fail: 0.038769 sec","job_status":"fail","queue_duration_s":0.041388,"scheduling_latency_s":0.041408,"redis_calls":7,"redis_duration_s":0.0030499999999999998,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002094,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038769,"completed_at":"2026-07-06T10:12:18.135Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.009952,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.897327] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.140Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b54abaf2c624857bacaabe31","created_at":"2026-07-06T10:12:17.579Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.584Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b54abaf2c624857bacaabe31: start","job_status":"start","queue_duration_s":0.555688,"scheduling_latency_s":0.55577} gitlab # [ 208.906387] gitaly[1385]: time="2026-07-06T10:12:18.424Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM5AM1ZARAPKMC9QMG90G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:48.222" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:18.423" grpc.time_ms=0.952 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.983738] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 62 gitlab # [ 208.984477] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.282Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b54abaf2c624857bacaabe31","created_at":"2026-07-06T10:12:17.579Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.584Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b54abaf2c624857bacaabe31: done: 0.142678 sec","job_status":"done","queue_duration_s":0.555688,"scheduling_latency_s":0.55577,"redis_calls":10,"redis_duration_s":0.004523,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000723,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002855,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.079,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.07,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000829994,"duration_s":0.142678,"completed_at":"2026-07-06T10:12:18.282Z","load_balancing_strategy":"primary","db_duration_s":0.078828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.057823] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.285Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"c951da95984cedc48d61f6e0","created_at":"2026-07-06T10:12:17.598Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.607Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-c951da95984cedc48d61f6e0: start","job_status":"start","queue_duration_s":0.677216,"scheduling_latency_s":0.677241} gitlab # [ 209.059118] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.319Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"c951da95984cedc48d61f6e0","created_at":"2026-07-06T10:12:17.598Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.607Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-c951da95984cedc48d61f6e0: done: 0.03467 sec","job_status":"done","queue_duration_s":0.677216,"scheduling_latency_s":0.677241,"redis_calls":13,"redis_duration_s":0.003414,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000703,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000492,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000460952,"duration_s":0.03467,"completed_at":"2026-07-06T10:12:18.319Z","load_balancing_strategy":"primary","db_duration_s":0.003524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.136504] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.321Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"9140b8c7b33a5f43da6a2ef0","created_at":"2026-07-06T10:12:17.609Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.617Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-9140b8c7b33a5f43da6a2ef0: start","job_status":"start","queue_duration_s":0.704182,"scheduling_latency_s":0.704212} gitlab # [ 209.137895] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.350Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"9140b8c7b33a5f43da6a2ef0","created_at":"2026-07-06T10:12:17.609Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.617Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-9140b8c7b33a5f43da6a2ef0: done: 0.029024 sec","job_status":"done","queue_duration_s":0.704182,"scheduling_latency_s":0.704212,"redis_calls":10,"redis_duration_s":0.00404,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000463,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000809321,"duration_s":0.029024,"completed_at":"2026-07-06T10:12:18.350Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.215553] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.354Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c4cae559908e6372def7c96e","created_at":"2026-07-06T10:12:17.618Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.626Z","job_size_bytes":19,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c4cae559908e6372def7c96e: start","job_status":"start","queue_duration_s":0.727208,"scheduling_latency_s":0.727754} gitlab # [ 209.217055] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.394Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c4cae559908e6372def7c96e","created_at":"2026-07-06T10:12:17.618Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.626Z","job_size_bytes":19,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c4cae559908e6372def7c96e: done: 0.040605 sec","job_status":"done","queue_duration_s":0.727208,"scheduling_latency_s":0.727754,"redis_calls":11,"redis_duration_s":0.0054719999999999994,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003328,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001593,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000687797,"duration_s":0.040605,"completed_at":"2026-07-06T10:12:18.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.286416] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.396Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7f8740f4d3a82812ee284aea","created_at":"2026-07-06T10:12:17.629Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5ba92b290e0434ffd4280a975048c221eb615aa5db8fd1f397623ac8e64389ed","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.635Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7f8740f4d3a82812ee284aea: start","job_status":"start","queue_duration_s":0.760929,"scheduling_latency_s":0.760955} gitlab # [ 209.360232] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.428Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7f8740f4d3a82812ee284aea","created_at":"2026-07-06T10:12:17.629Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5ba92b290e0434ffd4280a975048c221eb615aa5db8fd1f397623ac8e64389ed","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.635Z","job_size_bytes":65,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7f8740f4d3a82812ee284aea: done: 0.032 sec","job_status":"done","queue_duration_s":0.760929,"scheduling_latency_s":0.760955,"redis_calls":10,"redis_duration_s":0.002661,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000671,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000450616,"duration_s":0.032,"completed_at":"2026-07-06T10:12:18.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.430357] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.434Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"77f6ffd968b471cb189c0d1b","created_at":"2026-07-06T10:12:17.636Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.638Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-77f6ffd968b471cb189c0d1b: start","job_status":"start","queue_duration_s":0.796535,"scheduling_latency_s":0.796566} gitlab # [ 209.431728] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"77f6ffd968b471cb189c0d1b","created_at":"2026-07-06T10:12:17.636Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.638Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-77f6ffd968b471cb189c0d1b: done: 0.031955 sec","job_status":"done","queue_duration_s":0.796535,"scheduling_latency_s":0.796566,"redis_calls":7,"redis_duration_s":0.004422000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003422,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031955,"completed_at":"2026-07-06T10:12:18.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 209.500850] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.468Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"fdafaa0d2bb536ec4b5ff3a1","created_at":"2026-07-06T10:12:17.786Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.797Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-fdafaa0d2bb536ec4b5ff3a1: start","job_status":"start","queue_duration_s":0.670884,"scheduling_latency_s":0.670907} gitlab # [ 209.502158] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.496Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"fdafaa0d2bb536ec4b5ff3a1","created_at":"2026-07-06T10:12:17.786Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.797Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-fdafaa0d2bb536ec4b5ff3a1: done: 0.028657 sec","job_status":"done","queue_duration_s":0.670884,"scheduling_latency_s":0.670907,"redis_calls":10,"redis_duration_s":0.002453,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001562,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000395301,"duration_s":0.028657,"completed_at":"2026-07-06T10:12:18.496Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.09 seconds) gitlab # [ 209.570107] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.498Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"95ee81cfa52e54fb98bf6e34","created_at":"2026-07-06T10:12:17.798Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.811Z","job_size_bytes":6,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-95ee81cfa52e54fb98bf6e34: start","job_status":"start","queue_duration_s":0.685973,"scheduling_latency_s":0.685994} gitlab # [ 209.571109] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.517Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"95ee81cfa52e54fb98bf6e34","created_at":"2026-07-06T10:12:17.798Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.811Z","job_size_bytes":6,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-95ee81cfa52e54fb98bf6e34: done: 0.020015 sec","job_status":"done","queue_duration_s":0.685973,"scheduling_latency_s":0.685994,"redis_calls":10,"redis_duration_s":0.002301,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372114,"duration_s":0.020015,"completed_at":"2026-07-06T10:12:18.517Z","load_balancing_strategy":"primary","db_duration_s":0.00193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 209.644089] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"4705ea0993cb8c57927364ba","created_at":"2026-07-06T10:12:17.930Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.933Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4705ea0993cb8c57927364ba: start","job_status":"start","queue_duration_s":0.585984,"scheduling_latency_s":0.586003} gitlab # [ 209.718471] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:18.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"4705ea0993cb8c57927364ba","created_at":"2026-07-06T10:12:17.930Z","trace_propagation_headers":{"sentry-trace":"923fd37d5073437da65f4c60486b46c1-c368ef11aecb4934","baggage":"sentry-trace_id=923fd37d5073437da65f4c60486b46c1,sentry-release=Unknown"},"correlation_id":"01KWVEM42WN1KHQMKCBRMHE4GS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:17.933Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4705ea0993cb8c57927364ba: done: 0.065956 sec","job_status":"done","queue_duration_s":0.585984,"scheduling_latency_s":0.586003,"gitaly_calls":2,"gitaly_duration_s":0.006094,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003189,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00061,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000214,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.045905834,"duration_s":0.065956,"completed_at":"2026-07-06T10:12:18.585Z","load_balancing_strategy":"primary","db_duration_s":0.013184,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 209.792292] gitaly[1385]: time="2026-07-06T10:12:18.551Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:48.349" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:18.550" grpc.time_ms=0.935 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.793462] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 57 gitlab # [ 209.794313] gitaly[1385]: time="2026-07-06T10:12:18.575Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM42WN1KHQMKCBRMHE4GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:48.374" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:18.574" grpc.time_ms=0.536 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.795198] gitaly[1385]: time="2026-07-06T10:12:19.532Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM6D315GGD6JNNG47YX2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:49.330" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:19.531" grpc.time_ms=0.588 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 210.736930] gitaly[1385]: time="2026-07-06T10:12:20.670Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM7FMKPDJVT0WGJBTRKKR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:50.467" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:20.668" grpc.time_ms=1.22 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.825352] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 161 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.22 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 211.963896] gitaly[1385]: time="2026-07-06T10:12:21.897Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM8NPYWVD20G07VYAYDG8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:51.694" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:21.895" grpc.time_ms=1.307 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.037390] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 155 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.21 seconds) gitlab # [ 212.344393] nix-daemon[3457]: accepted connection from pid 3533, user nixbot gitlab # [ 212.376382] gitaly[1385]: time="2026-07-06T10:12:22.310Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:52.108" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:22.309" grpc.time_ms=1.004 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.460752] gitaly[1385]: time="2026-07-06T10:12:22.394Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:32.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:22.379" grpc.time_ms=14.529 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.576367] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:22.508Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5cfd17766960810e7a94b552","created_at":"2026-07-06T10:12:22.498Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-1f785d50db00421c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.505Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5cfd17766960810e7a94b552: start","job_status":"start","queue_duration_s":0.003334,"scheduling_latency_s":0.0034} gitlab # [ 212.600395] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:22 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/742a8404953aeb6f69baa0955481a1993fe6f49d HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 289 gitlab # [ 212.604543] nixbot[3042]: {"ts": "2026-07-06T10:12:22.535Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/742a8404953aeb6f69baa0955481a1993fe6f49d \"HTTP/1.1 201 Created\""} gitlab # [ 212.680989] gitaly[1385]: time="2026-07-06T10:12:22.615Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM9CEQ8EAWXQ3TVPPFP36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:52.212" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:22.613" grpc.time_ms=1.429 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.715248] gitaly[1385]: time="2026-07-06T10:12:22.649Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM9CEQ8EAWXQ3TVPPFP36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:32.272" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:22.642" grpc.time_ms=6.624 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.736663] gitaly[1385]: time="2026-07-06T10:12:22.669Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:22.668" grpc.time_ms=0.339 pid=1385 span.kind=server system=grpc gitlab # [ 212.752473] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:22 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/742a8404953aeb6f69baa0955481a1993fe6f49d HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 143 gitlab # [ 212.755282] nixbot[3042]: {"ts": "2026-07-06T10:12:22.689Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/742a8404953aeb6f69baa0955481a1993fe6f49d \"HTTP/1.1 201 Created\""} gitlab # [ 212.782642] gitaly[1385]: time="2026-07-06T10:12:22.716Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:22.716" grpc.time_ms=0.527 pid=1385 span.kind=server system=grpc gitlab # [ 212.822599] gitaly[1385]: time="2026-07-06T10:12:22.756Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:22.756" grpc.time_ms=0.436 pid=1385 span.kind=server system=grpc gitlab # [ 212.849428] gitaly[1385]: time="2026-07-06T10:12:22.783Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 diskcache=378d9ffa-6811-47fc-81fa-d28b8372b8d8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:52.432" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:22.633" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.864883] gitaly[1385]: time="2026-07-06T10:12:22.798Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=135 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7181 command.oublock=0 command.real_time_ms=141 command.spawn_token_fork_ms=0 command.system_time_ms=88 command.user_time_ms=47 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:52.432" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:22.633" grpc.time_ms=150.602 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.913157] gitaly[1385]: time="2026-07-06T10:12:22.847Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:52.645" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:22.846" grpc.time_ms=0.886 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.949665] gitaly[1385]: time="2026-07-06T10:12:22.883Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:52.681" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:22.882" grpc.time_ms=1.128 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 213.053466] gitaly[1385]: time="2026-07-06T10:12:22.986Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:52.783" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:22.984" grpc.time_ms=2.057 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.069974] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.003Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ffba470ac18399a2958eefd5","created_at":"2026-07-06T10:12:22.999Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"5cfd17766960810e7a94b552","job_size_bytes":3,"pid":2108,"job_status":"deduplicated","message":"PipelineProcessWorker JID-ffba470ac18399a2958eefd5: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 213.074222] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.008Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5cfd17766960810e7a94b552","created_at":"2026-07-06T10:12:22.498Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-1f785d50db00421c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.505Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2108,"job_status":"rescheduled","message":"PipelineProcessWorker JID-5cfd17766960810e7a94b552: rescheduled"} gitlab # [ 213.084325] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.017Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5cfd17766960810e7a94b552","created_at":"2026-07-06T10:12:22.498Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-1f785d50db00421c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.505Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5cfd17766960810e7a94b552: done: 0.508691 sec","job_status":"done","queue_duration_s":0.003334,"scheduling_latency_s":0.0034,"gitaly_calls":4,"gitaly_duration_s":0.202343,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.041137,"redis_read_bytes":262,"redis_write_bytes":23388,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002709,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.005003,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002097,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.008637,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14469,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.014891,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000678,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001204,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003209,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.211957,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.439448309,"duration_s":0.508691,"completed_at":"2026-07-06T10:12:23.017Z","load_balancing_strategy":"primary","db_duration_s":0.059463,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 213.162778] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.023Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ca46e84998a3a3f639a3f777","created_at":"2026-07-06T10:12:22.622Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.626Z","job_size_bytes":23,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ca46e84998a3a3f639a3f777: start","job_status":"start","queue_duration_s":0.39741,"scheduling_latency_s":0.397438} gitlab # [ 213.246392] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.061Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ca46e84998a3a3f639a3f777","created_at":"2026-07-06T10:12:22.622Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.626Z","job_size_bytes":23,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ca46e84998a3a3f639a3f777: done: 0.038064 sec","job_status":"done","queue_duration_s":0.39741,"scheduling_latency_s":0.397438,"redis_calls":7,"redis_duration_s":0.002613,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001956,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038064,"completed_at":"2026-07-06T10:12:23.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007676,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 213.248313] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.064Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"46e062f996fcb1e7954ad9b2","created_at":"2026-07-06T10:12:22.805Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.808Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-46e062f996fcb1e7954ad9b2: start","job_status":"start","queue_duration_s":0.255227,"scheduling_latency_s":0.255254} gitlab # [ 213.318329] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.086Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"46e062f996fcb1e7954ad9b2","created_at":"2026-07-06T10:12:22.805Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.808Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-46e062f996fcb1e7954ad9b2: done: 0.022426 sec","job_status":"done","queue_duration_s":0.255227,"scheduling_latency_s":0.255254,"redis_calls":7,"redis_duration_s":0.002636,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001927,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022426,"completed_at":"2026-07-06T10:12:23.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.386469] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.088Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c6b96b5715e6df3590cf2f9e","created_at":"2026-07-06T10:12:22.816Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.819Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c6b96b5715e6df3590cf2f9e: start","job_status":"start","queue_duration_s":0.268235,"scheduling_latency_s":0.268259} gitlab # [ 213.387940] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.117Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c6b96b5715e6df3590cf2f9e","created_at":"2026-07-06T10:12:22.816Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.819Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c6b96b5715e6df3590cf2f9e: done: 0.028918 sec","job_status":"done","queue_duration_s":0.268235,"scheduling_latency_s":0.268259,"redis_calls":7,"redis_duration_s":0.002548,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0019,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000648,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028918,"completed_at":"2026-07-06T10:12:23.117Z","load_balancing_strategy":"primary","db_duration_s":0.007816,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.456996] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.119Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5c21fb93a974547b929c334f","created_at":"2026-07-06T10:12:22.828Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.829Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5c21fb93a974547b929c334f: start","job_status":"start","queue_duration_s":0.289634,"scheduling_latency_s":0.28966} gitlab # [ 213.458323] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.146Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5c21fb93a974547b929c334f","created_at":"2026-07-06T10:12:22.828Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.829Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5c21fb93a974547b929c334f: done: 0.027055 sec","job_status":"done","queue_duration_s":0.289634,"scheduling_latency_s":0.28966,"redis_calls":10,"redis_duration_s":0.003593,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001054,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000645,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027055,"completed_at":"2026-07-06T10:12:23.146Z","load_balancing_strategy":"primary","db_duration_s":0.004306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.542151] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.148Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"213aeb9cd0828bbfcb8b4e7b","created_at":"2026-07-06T10:12:22.830Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.831Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-213aeb9cd0828bbfcb8b4e7b: start","job_status":"start","queue_duration_s":0.316358,"scheduling_latency_s":0.316382} gitlab # [ 213.543382] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.184Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"213aeb9cd0828bbfcb8b4e7b","created_at":"2026-07-06T10:12:22.830Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.831Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-213aeb9cd0828bbfcb8b4e7b: done: 0.035745 sec","job_status":"done","queue_duration_s":0.316358,"scheduling_latency_s":0.316382,"redis_calls":7,"redis_duration_s":0.003465,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002643,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000822,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035745,"completed_at":"2026-07-06T10:12:23.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.622525] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.187Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ea3ec70e79ce0135be2e8fe9","created_at":"2026-07-06T10:12:22.832Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.835Z","job_size_bytes":19,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ea3ec70e79ce0135be2e8fe9: start","job_status":"start","queue_duration_s":0.351373,"scheduling_latency_s":0.351791} gitlab # [ 213.632431] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.209Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ea3ec70e79ce0135be2e8fe9","created_at":"2026-07-06T10:12:22.832Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.835Z","job_size_bytes":19,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ea3ec70e79ce0135be2e8fe9: done: 0.022461 sec","job_status":"done","queue_duration_s":0.351373,"scheduling_latency_s":0.351791,"redis_calls":8,"redis_duration_s":0.0018869999999999998,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001393,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000494,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022461,"completed_at":"2026-07-06T10:12:23.208Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.702281] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.210Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ab4654c519663bfdaa227f24","created_at":"2026-07-06T10:12:22.836Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.837Z","job_size_bytes":66,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ab4654c519663bfdaa227f24: start","job_status":"start","queue_duration_s":0.373008,"scheduling_latency_s":0.37303} gitlab # [ 213.782341] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.232Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ab4654c519663bfdaa227f24","created_at":"2026-07-06T10:12:22.836Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.837Z","job_size_bytes":66,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ab4654c519663bfdaa227f24: done: 0.021706 sec","job_status":"done","queue_duration_s":0.373008,"scheduling_latency_s":0.37303,"redis_calls":7,"redis_duration_s":0.002353,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001728,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021706,"completed_at":"2026-07-06T10:12:23.232Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001773,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.849344] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.234Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"e3cdcfe3622512a3d3c4dd8b","created_at":"2026-07-06T10:12:22.838Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.840Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e3cdcfe3622512a3d3c4dd8b: start","job_status":"start","queue_duration_s":0.393938,"scheduling_latency_s":0.393959} gitlab # [ 213.850795] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.258Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"e3cdcfe3622512a3d3c4dd8b","created_at":"2026-07-06T10:12:22.838Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.840Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e3cdcfe3622512a3d3c4dd8b: done: 0.024519 sec","job_status":"done","queue_duration_s":0.393938,"scheduling_latency_s":0.393959,"redis_calls":7,"redis_duration_s":0.002432,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000707,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024519,"completed_at":"2026-07-06T10:12:23.258Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004068,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.921367] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.260Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cec6fce789fcb48e5f537da2","created_at":"2026-07-06T10:12:22.914Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.918Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cec6fce789fcb48e5f537da2: start","job_status":"start","queue_duration_s":0.342208,"scheduling_latency_s":0.342267} gitlab # [ 213.922786] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.280Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cec6fce789fcb48e5f537da2","created_at":"2026-07-06T10:12:22.914Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.918Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cec6fce789fcb48e5f537da2: done: 0.019759 sec","job_status":"done","queue_duration_s":0.342208,"scheduling_latency_s":0.342267,"redis_calls":7,"redis_duration_s":0.00241,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000711,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019759,"completed_at":"2026-07-06T10:12:23.280Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004158,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 213.998505] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.281Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"840d5be1c351a1a7b6876301","created_at":"2026-07-06T10:12:22.919Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.922Z","job_size_bytes":6,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-840d5be1c351a1a7b6876301: start","job_status":"start","queue_duration_s":0.35955,"scheduling_latency_s":0.359572} gitlab # [ 213.999956] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.295Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"840d5be1c351a1a7b6876301","created_at":"2026-07-06T10:12:22.919Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.922Z","job_size_bytes":6,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-840d5be1c351a1a7b6876301: done: 0.013493 sec","job_status":"done","queue_duration_s":0.35955,"scheduling_latency_s":0.359572,"redis_calls":7,"redis_duration_s":0.002541,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013493,"completed_at":"2026-07-06T10:12:23.295Z","load_balancing_strategy":"primary","db_duration_s":0.000549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 214.072210] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.296Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2303a52bbf425b4d2e7e78b3","created_at":"2026-07-06T10:12:22.923Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.930Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2303a52bbf425b4d2e7e78b3: start","job_status":"start","queue_duration_s":0.366853,"scheduling_latency_s":0.366873} gitlab # [ 214.077171] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.323Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2303a52bbf425b4d2e7e78b3","created_at":"2026-07-06T10:12:22.923Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:22.930Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2303a52bbf425b4d2e7e78b3: done: 0.026338 sec","job_status":"done","queue_duration_s":0.366853,"scheduling_latency_s":0.366873,"redis_calls":10,"redis_duration_s":0.003836,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000692,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000436368,"duration_s":0.026338,"completed_at":"2026-07-06T10:12:23.323Z","load_balancing_strategy":"primary","db_duration_s":0.004991,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 214.164082] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.324Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"254dd460e14277edadb29d98","created_at":"2026-07-06T10:12:23.008Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:23.010Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-254dd460e14277edadb29d98: start","job_status":"start","queue_duration_s":0.314508,"scheduling_latency_s":0.314528} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 1.27 seconds) gitlab # [ 214.166727] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.440Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"254dd460e14277edadb29d98","created_at":"2026-07-06T10:12:23.008Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-94a5416d1adb4c7c","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:23.010Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-254dd460e14277edadb29d98: done: 0.11565 sec","job_status":"done","queue_duration_s":0.314508,"scheduling_latency_s":0.314528,"gitaly_calls":3,"gitaly_duration_s":0.010667,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010128,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001602,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002161,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000397,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002049,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000405,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000703,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001209,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068547,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.094696952,"duration_s":0.11565,"completed_at":"2026-07-06T10:12:23.440Z","load_balancing_strategy":"primary","db_duration_s":0.033516,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 214.255182] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.441Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"cf3ca2a5b2868a09e55e5d1a","created_at":"2026-07-06T10:12:23.368Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-23b9dfcfa8c24ea7","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:23.370Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-cf3ca2a5b2868a09e55e5d1a: start","job_status":"start","queue_duration_s":0.071481,"scheduling_latency_s":0.071503} gitlab # [ 214.321881] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:23.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"cf3ca2a5b2868a09e55e5d1a","created_at":"2026-07-06T10:12:23.368Z","trace_propagation_headers":{"sentry-trace":"8f389a6ea1e445f6a379b89154071ebc-23b9dfcfa8c24ea7","baggage":"sentry-trace_id=8f389a6ea1e445f6a379b89154071ebc,sentry-release=Unknown"},"correlation_id":"01KWVEM934FJH0GY5M36Y0HPS3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:23.370Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-cf3ca2a5b2868a09e55e5d1a: done: 0.024182 sec","job_status":"done","queue_duration_s":0.071481,"scheduling_latency_s":0.071503,"redis_calls":7,"redis_duration_s":0.001687,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001194,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024182,"completed_at":"2026-07-06T10:12:23.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 214.392558] gitaly[1385]: time="2026-07-06T10:12:23.380Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:53.178" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:23.378" grpc.time_ms=1.24 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.394238] gitlab-workhorse[1287]: gitlab 127.0.0.1 - - [2026/07/06:10:12:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 68 gitlab # [ 214.394980] gitaly[1385]: time="2026-07-06T10:12:23.404Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:53.202" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:23.403" grpc.time_ms=0.863 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.396213] gitaly[1385]: time="2026-07-06T10:12:23.430Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEM934FJH0GY5M36Y0HPS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:53.228" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:23.429" grpc.time_ms=0.766 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.396953] gitaly[1385]: time="2026-07-06T10:12:24.202Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMAZ5FJ1TS18KZPEG2BQ7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:54.000" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:24.201" grpc.time_ms=0.805 pid=1385 remote_ip=127.0.0.1 span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 215.452711] gitaly[1385]: time="2026-07-06T10:12:25.385Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMC2FSV63B96373S619NA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:55.183" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:25.383" grpc.time_ms=1.275 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.503184] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 140 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.20 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 216.628166] gitaly[1385]: time="2026-07-06T10:12:26.561Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMD7RJEHK8V5WHE9VQDKD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:56.159" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:26.559" grpc.time_ms=0.955 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.676844] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 120 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.17 seconds) gitlab # [ 216.858739] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:26.791Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"bd51762eb4c0c81aeb47d736","created_at":"2026-07-06T10:12:00.899Z","trace_propagation_headers":{"sentry-trace":"143fd74c70d942feb1991a75c89d40a3-ab4aed4f7f764553","baggage":"sentry-trace_id=143fd74c70d942feb1991a75c89d40a3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKKZCPXGWG49TX528EN3Q","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:2f00f5c363334f1952d8a1f06fcfca9c32c947c3cd9d07b928321c990afbcbcd","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:26.788Z","retry_count":0,"job_size_bytes":451,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-bd51762eb4c0c81aeb47d736: start","job_status":"start","queue_duration_s":0.00332,"scheduling_latency_s":0.003493} gitlab # [ 217.054643] sidekiq-cluster[2108]: {"severity":"WARN","time":"2026-07-06T10:12:26.988Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"bd51762eb4c0c81aeb47d736","created_at":"2026-07-06T10:12:00.899Z","trace_propagation_headers":{"sentry-trace":"143fd74c70d942feb1991a75c89d40a3-ab4aed4f7f764553","baggage":"sentry-trace_id=143fd74c70d942feb1991a75c89d40a3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEKKZCPXGWG49TX528EN3Q","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:2f00f5c363334f1952d8a1f06fcfca9c32c947c3cd9d07b928321c990afbcbcd","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:26.788Z","retry_count":0,"job_size_bytes":451,"pid":2108,"sidekiq_tid":"mabc","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-bd51762eb4c0c81aeb47d736: fail: 0.196169 sec","job_status":"fail","queue_duration_s":0.00332,"scheduling_latency_s":0.003493,"redis_calls":7,"redis_duration_s":0.0044340000000000004,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001662,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.196169,"completed_at":"2026-07-06T10:12:26.987Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.008837,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 217.680967] nix-daemon[3457]: accepted connection from pid 3603, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 217.767604] gitaly[1385]: time="2026-07-06T10:12:27.701Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC3AV6QPZ3M7TD9V6K5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:57.299" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:27.699" grpc.time_ms=1.111 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.788447] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 70 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.11 seconds) gitlab # [ 217.813730] gitaly[1385]: time="2026-07-06T10:12:27.747Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:57.546" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:27.746" grpc.time_ms=0.769 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.870795] gitaly[1385]: time="2026-07-06T10:12:27.804Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T10:12:37.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:27.794" grpc.time_ms=10.127 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.919240] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:27.852Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d873aa7cc81d8c3bd4adf320","created_at":"2026-07-06T10:12:27.846Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-fb11064898de4109","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:27.849Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d873aa7cc81d8c3bd4adf320: start","job_status":"start","queue_duration_s":0.002648,"scheduling_latency_s":0.002713} gitlab # [ 217.930908] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/742a8404953aeb6f69baa0955481a1993fe6f49d HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 216 gitlab # [ 217.932790] nixbot[3042]: {"ts": "2026-07-06T10:12:27.866Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/742a8404953aeb6f69baa0955481a1993fe6f49d \"HTTP/1.1 201 Created\""} gitlab # [ 218.046339] gitaly[1385]: time="2026-07-06T10:12:27.980Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:27.979" grpc.time_ms=0.624 pid=1385 span.kind=server system=grpc gitlab # [ 218.092724] gitaly[1385]: time="2026-07-06T10:12:28.026Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:28.026" grpc.time_ms=0.388 pid=1385 span.kind=server system=grpc gitlab # [ 218.127627] nix-daemon[3457]: accepted connection from pid 3642, user nixbot gitlab # [ 218.130889] gitaly[1385]: time="2026-07-06T10:12:28.065Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T10:12:28.064" grpc.time_ms=0.296 pid=1385 span.kind=server system=grpc gitlab # [ 218.149460] gitaly[1385]: time="2026-07-06T10:12:28.083Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:57.745" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:27.945" operations="[repack_refs]" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.151373] gitaly[1385]: time="2026-07-06T10:12:28.085Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:57.745" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:27.945" pid=1385 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-07-06 10:12:02.59117246 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 218.165754] gitaly[1385]: time="2026-07-06T10:12:28.099Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:57.745" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:27.945" pid=1385 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 218.166842] gitaly[1385]: time="2026-07-06T10:12:28.100Z" level=info msg="optimized repository" clean_stale_data_time=13.930542ms clean_work_trees_time="94.984µs" commit_graph_time="7.823µs" correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.102µs" pid=1385 prune_time="7.543µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.381µs" storage_name=default system=housekeeping total_time=14.259634ms gitlab # [ 218.167879] gitaly[1385]: time="2026-07-06T10:12:28.100Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:57.745" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:27.945" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.235216] gitaly[1385]: time="2026-07-06T10:12:28.169Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 diskcache=e70f4590-12c2-444f-8b85-b1f9027c4a0b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:57.745" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:27.945" pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.236690] gitaly[1385]: time="2026-07-06T10:12:28.169Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=128 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=8549 command.oublock=0 command.real_time_ms=146 command.spawn_token_fork_ms=3 command.system_time_ms=88 command.user_time_ms=40 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T10:12:57.745" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T10:12:27.945" grpc.time_ms=223.364 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.302102] gitaly[1385]: time="2026-07-06T10:12:28.234Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVEMEWZHYM8AAB1C5P91K3B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:28.227" grpc.time_ms=7.025 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.303462] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 44 gitlab # [ 218.377134] gitaly[1385]: time="2026-07-06T10:12:28.311Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEZ6RGXPRV8AEXZ31WY2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:28.300" pid=1385 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab # [ 218.396135] gitaly[1385]: time="2026-07-06T10:12:28.311Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEZ6RGXPRV8AEXZ31WY2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:28.300" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.417670] gitaly[1385]: time="2026-07-06T10:12:28.351Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEZ6RGXPRV8AEXZ31WY2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T10:12:28.300" grpc.time_ms=51.072 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.419450] gitlab-workhorse[1287]: time="2026-07-06T10:12:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 218.421136] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 91 gitlab # [ 218.477742] gitaly[1385]: time="2026-07-06T10:12:28.411Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T10:12:58.210" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:28.410" grpc.time_ms=1.04 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.526869] gitaly[1385]: time="2026-07-06T10:12:28.461Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:58.259" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:28.459" grpc.time_ms=1.195 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.582741] gitaly[1385]: time="2026-07-06T10:12:28.516Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMEC0CNEZFZEP868Z6HG9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:58.315" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:28.515" grpc.time_ms=0.818 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.602434] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.534Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d873aa7cc81d8c3bd4adf320","created_at":"2026-07-06T10:12:27.846Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-fb11064898de4109","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:27.849Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d873aa7cc81d8c3bd4adf320: done: 0.682056 sec","job_status":"done","queue_duration_s":0.002648,"scheduling_latency_s":0.002713,"gitaly_calls":4,"gitaly_duration_s":0.25645,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.031393,"redis_read_bytes":173,"redis_write_bytes":20930,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001358,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.006074,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.007871,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13383,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.008507,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000336,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00084,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.005049,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.179,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.156,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.166138,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.640721656,"duration_s":0.682056,"completed_at":"2026-07-06T10:12:28.534Z","load_balancing_strategy":"primary","db_duration_s":0.181673,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 218.613240] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.535Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"40e2aaa20ec3be315e255f57","created_at":"2026-07-06T10:12:27.936Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:27.938Z","job_size_bytes":23,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-40e2aaa20ec3be315e255f57: start","job_status":"start","queue_duration_s":0.597339,"scheduling_latency_s":0.597361} gitlab # [ 218.728628] nix-daemon[3457]: accepted connection from pid 3683, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses gitlab # [ 218.771383] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.704Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"40e2aaa20ec3be315e255f57","created_at":"2026-07-06T10:12:27.936Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:27.938Z","job_size_bytes":23,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-40e2aaa20ec3be315e255f57: done: 0.168904 sec","job_status":"done","queue_duration_s":0.597339,"scheduling_latency_s":0.597361,"redis_calls":7,"redis_duration_s":0.004098,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003314,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.168904,"completed_at":"2026-07-06T10:12:28.704Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010109,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 218.774966] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.708Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1779dad1f5808040380d7315","created_at":"2026-07-06T10:12:28.174Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:28.176Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1779dad1f5808040380d7315: start","job_status":"start","queue_duration_s":0.53092,"scheduling_latency_s":0.530975} gitlab # [ 218.811237] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.744Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1779dad1f5808040380d7315","created_at":"2026-07-06T10:12:28.174Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:28.176Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1779dad1f5808040380d7315: done: 0.036991 sec","job_status":"done","queue_duration_s":0.53092,"scheduling_latency_s":0.530975,"redis_calls":7,"redis_duration_s":0.002556,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036991,"completed_at":"2026-07-06T10:12:28.744Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002748,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.887342] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.747Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9d8ae9c914816d617df881cd","created_at":"2026-07-06T10:12:28.185Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:28.188Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9d8ae9c914816d617df881cd: start","job_status":"start","queue_duration_s":0.558992,"scheduling_latency_s":0.55903} gitlab # [ 218.923236] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.856Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9d8ae9c914816d617df881cd","created_at":"2026-07-06T10:12:28.185Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:28.188Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9d8ae9c914816d617df881cd: done: 0.109185 sec","job_status":"done","queue_duration_s":0.558992,"scheduling_latency_s":0.55903,"redis_calls":7,"redis_duration_s":0.001908,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001419,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.082,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.082,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.109185,"completed_at":"2026-07-06T10:12:28.856Z","load_balancing_strategy":"primary","db_duration_s":0.082379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.997562] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.864Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4082cf05fa37ef0382215f8f","created_at":"2026-07-06T10:12:28.329Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:28.382Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4082cf05fa37ef0382215f8f: start","job_status":"start","queue_duration_s":0.481776,"scheduling_latency_s":0.481803} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses, in 0.28 seconds) (finished: subtest: eval and build post commit statuses, in 11.90 seconds) (finished: run the VM test script, in 258.64 seconds) gitlab # [ 218.999282] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.924Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4082cf05fa37ef0382215f8f","created_at":"2026-07-06T10:12:28.329Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:28.382Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4082cf05fa37ef0382215f8f: done: 0.059476 sec","job_status":"done","queue_duration_s":0.481776,"scheduling_latency_s":0.481803,"redis_calls":10,"redis_duration_s":0.003672,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002451,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000691,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059476,"completed_at":"2026-07-06T10:12:28.924Z","load_balancing_strategy":"primary","db_duration_s":0.004491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.088585] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.925Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"42b98075308c24131b44bf1b","created_at":"2026-07-06T10:12:28.387Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:28.390Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-42b98075308c24131b44bf1b: start","job_status":"start","queue_duration_s":0.535301,"scheduling_latency_s":0.535326} gitlab # [ 219.090462] gitaly[1385]: time="2026-07-06T10:12:28.984Z" level=info msg="finished unary call with code OK" catfile.duration_ms=101 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=101 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVEMFER83FR9PPE8AERVYZ2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T10:12:58.480" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T10:12:28.881" grpc.time_ms=102.188 pid=1385 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.091574] gitlab-workhorse[1287]: gitlab 2001:db8:1::1 - - [2026/07/06:10:12:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/742a8404953aeb6f69baa0955481a1993fe6f49d/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 250 test script finished in 258.78s cleanup kill QemuMachine (pid 13) gitlab # [ 219.092137] sidekiq-cluster[2108]: {"severity":"INFO","time":"2026-07-06T10:12:28.952Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"42b98075308c24131b44bf1b","created_at":"2026-07-06T10:12:28.387Z","trace_propagation_headers":{"sentry-trace":"f9c586cab2054b6d8791f5351f99be07-10b27cae1dc24949","baggage":"sentry-trace_id=f9c586cab2054b6d8791f5351f99be07,sentry-release=Unknown"},"correlation_id":"01KWVEMEC0CNEZFZEP868Z6HG9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-06T10:12:28.390Z","job_size_bytes":3,"pid":2108,"sidekiq_tid":"m7w8","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-42b98075308c24131b44bf1b: done: 0.026648 sec","job_status":"done","queue_duration_s":0.535301,"scheduling_latency_s":0.535326,"redis_calls":7,"redis_duration_s":0.004952,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00394,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_durqemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13) gitlab # a (finished: cleanup, in 1.47 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-06T10:12:12.791Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-06T10:12:15.704908Z', 'started_at': '2026-07-06T10:12:16.152273Z', 'finished_at': None}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': '2026-07-06T10:12:22.479Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T10:12:22.665Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': '2026-07-06T10:12:22.479Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T10:12:22.665Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': '2026-07-06T10:12:22.479Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T10:12:22.665Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': '2026-07-06T10:12:22.479Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T10:12:22.665Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T10:12:16.110Z', 'started_at': None, 'finished_at': '2026-07-06T10:12:22.479Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '742a8404953aeb6f69baa0955481a1993fe6f49d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-06T10:12:22.665Z', 'started_at': None, 'finished_at': '2026-07-06T10:12:27.837Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]