tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.wGcJqhpRQ4', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: ba7c3a22-777e-4c61-ba24-2f33c4003f0f (finished: start all VMs, in 64.73 seconds) gitlab # Superblock backups stored on blocks: gitlab: waiting for unit setup-gitlab.service gitlab # 32768, 98304, 163840, 229376 gitlab: waiting for the VM to finish booting gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b6wl5763gmsvvwqv0qnqjb9nag9ipsgs-nixos-system-gitlab-test/init regInfo=/nix/store/5wkbmmb7dq60vvdf4q8jixzy5mky2zqz-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 451044437 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor gitlab # [ 0.000805] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000835] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000880] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002731] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002743] Using GB pages for direct mapping gitlab # [ 0.002834] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002841] ACPI: Early table checksum verification disabled gitlab # [ 0.002844] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002848] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002852] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002859] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002861] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002863] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002870] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002872] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002873] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002874] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002874] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002875] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002875] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002876] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003088] No NUMA configuration found gitlab # [ 0.003089] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003092] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.005244] Zone ranges: gitlab # [ 0.005245] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005247] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005248] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005249] Device empty gitlab # [ 0.005249] Movable zone start for each node gitlab # [ 0.005250] Early memory node ranges gitlab # [ 0.005250] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005251] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005252] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005253] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005273] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005526] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.114336] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.114757] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.114771] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.114802] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.114805] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.114807] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.114808] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.114809] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.114809] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.114812] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.114813] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.114818] TSC deadline timer available gitlab # [ 0.114823] CPU topo: Max. logical packages: 1 gitlab # [ 0.114823] CPU topo: Max. logical dies: 1 gitlab # [ 0.114824] CPU topo: Max. dies per package: 1 gitlab # [ 0.114828] CPU topo: Max. threads per core: 1 gitlab # [ 0.114828] CPU topo: Num. cores per package: 4 gitlab # [ 0.114829] CPU topo: Num. threads per package: 4 gitlab # [ 0.114829] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.114852] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.114876] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.114879] kvm-guest: setup PV sched yield gitlab # [ 0.114896] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.114897] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.114898] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.114900] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.114901] Booting paravirtualized kernel on KVM gitlab # [ 0.114905] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.119372] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.121621] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.121698] kvm-guest: PV spinlocks enabled gitlab # [ 0.121700] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.121703] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b6wl5763gmsvvwqv0qnqjb9nag9ipsgs-nixos-system-gitlab-test/init regInfo=/nix/store/5wkbmmb7dq60vvdf4q8jixzy5mky2zqz-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.121798] Unknown kernel command line parameters "regInfo=/nix/store/5wkbmmb7dq60vvdf4q8jixzy5mky2zqz-closure-info/registration", will be passed to user space. gitlab # [ 0.121811] random: crng init done gitlab # [ 0.121812] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.130877] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.135295] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.135371] software IO TLB: area num 4. gitlab # [ 0.208487] Fallback order for Node 0: 0 gitlab # [ 0.208494] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.208496] Policy zone: Normal gitlab # [ 0.210885] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.220455] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.233340] allocated 12582912 bytes of page_ext gitlab # [ 0.243336] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.243338] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.244221] Dynamic Preempt: lazy gitlab # [ 0.244420] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.244421] rcu: RCU event tracing is enabled. gitlab # [ 0.244421] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.244423] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.244424] Rude variant of Tasks RCU enabled. gitlab # [ 0.244424] Tracing variant of Tasks RCU enabled. gitlab # [ 0.244425] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.244425] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.244454] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.244457] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.244459] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.248861] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.249193] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.249202] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.249312] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.252904] Console: colour VGA+ 80x25 gitlab # [ 0.252908] printk: legacy console [tty0] enabled gitlab # [ 0.283231] printk: legacy console [ttyS0] enabled gitlab # [ 0.391189] ACPI: Core revision 20250807 gitlab # [ 0.392084] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.393691] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.394756] x2apic enabled gitlab # [ 0.395519] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.396473] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.397743] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.399245] kvm-guest: setup PV IPIs gitlab # [ 0.400858] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.401925] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.403716] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) gitlab # [ 0.404798] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.405822] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.407713] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.408717] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.409714] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.410713] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.411713] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.412713] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.413713] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.415720] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.416714] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.417714] active return thunk: srso_alias_return_thunk gitlab # [ 0.418734] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.419713] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.421539] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.422536] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.423545] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.424713] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.425713] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.426713] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.427713] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.428713] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.429713] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.430713] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.432501] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.462633] Freeing SMP alternatives memory: 44K gitlab # [ 0.463335] pid_max: default: 32768 minimum: 301 gitlab # [ 0.463817] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.464808] landlock: Up and running. gitlab # [ 0.466185] Yama: becoming mindful. gitlab # [ 0.466935] LSM support for eBPF active gitlab # [ 0.467927] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.468847] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.471970] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.473311] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.473722] ... version: 2 gitlab # [ 0.474488] ... bit width: 48 gitlab # [ 0.474715] ... generic counters: 6 gitlab # [ 0.475467] ... generic bitmap: 000000000000003f gitlab # [ 0.475715] ... fixed-purpose counters: 0 gitlab # [ 0.476504] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.476715] ... value mask: 0000ffffffffffff gitlab # [ 0.477663] ... max period: 00007fffffffffff gitlab # [ 0.478458] ... global_ctrl mask: 000000000000003f gitlab # [ 0.478831] signal: max sigframe size: 3376 gitlab # [ 0.479665] rcu: Hierarchical SRCU implementation. gitlab # [ 0.480371] rcu: Max phase no-delay instances is 400. gitlab # [ 0.480896] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.486369] smp: Bringing up secondary CPUs ... gitlab # [ 0.487094] smpboot: x86: Booting SMP configuration: gitlab # [ 0.487719] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.490812] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.492452] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) gitlab # [ 0.493162] Memory: 6011572K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved) gitlab # [ 0.494716] devtmpfs: initialized gitlab # [ 0.495604] x86/mm: Memory block size: 128MB gitlab # [ 0.498842] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.499789] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.500807] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.502075] PM: RTC time: 16:05:20, date: 2026-09-06 gitlab # [ 0.505767] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.508075] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.508770] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.510805] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.511728] audit: initializing netlink subsys (disabled) gitlab # [ 0.512766] audit: type=2000 audit(1788710721.459:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.513019] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.514716] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.515716] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.516715] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.517715] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.518730] cpuidle: using governor menu gitlab # [ 0.520915] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.521957] PCI: Using configuration type 1 for base access gitlab # [ 0.522717] PCI: Using configuration type 1 for extended access gitlab # [ 0.523861] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.526109] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.527633] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.528598] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.529597] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.534151] ACPI: Added _OSI(Module Device) gitlab # [ 0.536725] ACPI: Added _OSI(Processor Device) gitlab # [ 0.538722] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.545559] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.552171] ACPI: Interpreter enabled gitlab # [ 0.553802] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.555722] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.557985] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.561721] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.564170] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.576882] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.579767] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.582572] acpiphp: Slot [3] registered gitlab # [ 0.584763] acpiphp: Slot [4] registered gitlab # [ 0.585762] acpiphp: Slot [5] registered gitlab # [ 0.586759] acpiphp: Slot [6] registered gitlab # [ 0.588758] acpiphp: Slot [7] registered gitlab # [ 0.589776] acpiphp: Slot [8] registered gitlab # [ 0.591760] acpiphp: Slot [9] registered gitlab # [ 0.592759] acpiphp: Slot [10] registered gitlab # [ 0.593758] acpiphp: Slot [11] registered gitlab # [ 0.595776] acpiphp: Slot [12] registered gitlab # [ 0.596760] acpiphp: Slot [13] registered gitlab # [ 0.598757] acpiphp: Slot [14] registered gitlab # [ 0.599756] acpiphp: Slot [15] registered gitlab # [ 0.600496] acpiphp: Slot [16] registered gitlab # [ 0.600736] acpiphp: Slot [17] registered gitlab # [ 0.601482] acpiphp: Slot [18] registered gitlab # [ 0.602766] acpiphp: Slot [19] registered gitlab # [ 0.603494] acpiphp: Slot [20] registered gitlab # [ 0.603736] acpiphp: Slot [21] registered gitlab # [ 0.604501] acpiphp: Slot [22] registered gitlab # [ 0.605744] acpiphp: Slot [23] registered gitlab # [ 0.606464] acpiphp: Slot [24] registered gitlab # [ 0.606735] acpiphp: Slot [25] registered gitlab # [ 0.607497] acpiphp: Slot [26] registered gitlab # [ 0.608778] acpiphp: Slot [27] registered gitlab # [ 0.609508] acpiphp: Slot [28] registered gitlab # [ 0.609736] acpiphp: Slot [29] registered gitlab # [ 0.610516] acpiphp: Slot [30] registered gitlab # [ 0.611744] acpiphp: Slot [31] registered gitlab # [ 0.612480] PCI host bridge to bus 0000:00 gitlab # [ 0.612737] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.613717] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.614716] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.616716] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.617715] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.618728] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.619911] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.622175] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.624139] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.626744] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.627691] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.628665] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.629524] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.631594] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.632741] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.634768] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.636617] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.638986] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.639722] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.640956] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.654782] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.655729] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.656737] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.657894] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.659454] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.663725] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.664621] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.665524] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.678700] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.683682] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.694740] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.696737] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.699777] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.705799] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.715742] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.717737] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.719775] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.725787] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.732737] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.734731] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.736762] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.741253] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.748758] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.749716] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.750778] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.759061] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.769758] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.771739] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.774778] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.780141] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.783754] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.784724] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.785739] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.787734] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.790154] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.792729] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.793738] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.796991] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.799754] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.800629] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.801562] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.803991] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.807747] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.808633] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.809477] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.814781] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.815902] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.816865] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.817864] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.819792] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.821995] iommu: Default domain type: Translated gitlab # [ 0.822717] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.824060] ACPI: bus type USB registered gitlab # [ 0.824789] usbcore: registered new interface driver usbfs gitlab # [ 0.825735] usbcore: registered new interface driver hub gitlab # [ 0.826642] usbcore: registered new device driver usb gitlab # [ 0.828537] NetLabel: Initializing gitlab # [ 0.829122] NetLabel: domain hash size = 128 gitlab # [ 0.830715] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.831746] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.832698] PCI: Using ACPI for IRQ routing gitlab # [ 0.833871] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.834712] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.834712] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.836720] vgaarb: loaded gitlab # [ 0.837486] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.838308] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.843923] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.846876] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.847567] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.849021] pnp: PnP ACPI init gitlab # [ 0.850189] pnp: PnP ACPI: found 6 devices gitlab # [ 0.858717] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.860189] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.861221] NET: Registered PF_INET protocol family gitlab # [ 0.863250] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.881463] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.882956] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.884285] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.886832] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.888130] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.889693] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.891200] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.892607] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.893895] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.894809] NET: Registered PF_XDP protocol family gitlab # [ 0.895628] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 0.896602] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 0.897575] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 0.898635] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.899709] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.900978] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 0.901909] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 0.904220] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 0.906709] PCI: CLS 0 bytes, default 64 gitlab # [ 0.907425] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 0.907689] Trying to unpack rootfs image as initramfs... gitlab # [ 0.908450] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 0.910790] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.931156] Initialise system trusted keyrings gitlab # [ 0.932174] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 0.943458] Key type asymmetric registered gitlab # [ 0.944184] Asymmetric key parser 'x509' registered gitlab # [ 0.945034] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 0.946612] io scheduler mq-deadline registered gitlab # [ 0.947364] io scheduler kyber registered gitlab # [ 0.950851] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.952114] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 0.954294] Linux agpgart interface v0.103 gitlab # [ 0.955043] ACPI: bus type drm_connector registered gitlab # [ 0.957828] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.958844] usbserial: USB Serial support registered for generic gitlab # [ 0.959778] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 0.959778] Please enable it if your BIOS has the CPPC option. gitlab # [ 0.961938] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 0.963282] drop_monitor: Initializing network drop monitor service gitlab # [ 0.964438] NET: Registered PF_INET6 protocol family gitlab # [ 0.966036] Segment Routing with IPv6 gitlab # [ 0.966681] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.968851] IPI shorthand broadcast: enabled gitlab # [ 0.973167] sched_clock: Marking stable (826011069, 146510877)->(996365078, -23843132) gitlab # [ 0.974834] registered taskstats version 1 gitlab # [ 0.975872] Loading compiled-in X.509 certificates gitlab # [ 0.985726] Demotion targets for Node 0: null gitlab # [ 0.986758] Key type .fscrypt registered gitlab # [ 0.987419] Key type fscrypt-provisioning registered gitlab # [ 0.988358] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.989255] ima: Allocated hash algorithm: sha1 gitlab # [ 0.990109] ima: No architecture policies found gitlab # [ 0.991141] PM: Magic number: 10:678:84 gitlab # [ 0.992748] RAS: Correctable Errors collector initialized. gitlab # [ 0.997223] clk: Disabling unused clocks gitlab # [ 0.997932] PM: genpd: Disabling unused power domains gitlab # [ 1.169592] Freeing initrd memory: 29392K gitlab # [ 1.176451] Freeing unused decrypted memory: 2028K gitlab # [ 1.184501] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.187276] Write protecting the kernel read-only data: 32768k gitlab # [ 1.192079] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.195897] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.264181] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.265334] Run /init as init process gitlab # [ 1.278174] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.300196] fuse: init (API version 7.45) gitlab # [ 1.308561] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.315401] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.318494] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.360074] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.696882] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.737237] systemd[1]: Detected virtualization kvm. gitlab # [ 1.739462] systemd[1]: Detected architecture x86-64. gitlab # [ 1.741631] systemd[1]: Running in initrd. gitlab # [ 1.744071] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.747147] systemd[1]: Hostname set to . gitlab # [ 1.865871] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.924896] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.933273] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.934579] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.937221] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.939391] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.941547] systemd[1]: Reached target Path Units. gitlab # [ 1.943190] systemd[1]: Reached target Slice Units. gitlab # [ 1.944873] systemd[1]: Reached target Swaps. gitlab # [ 1.946394] systemd[1]: Reached target Timer Units. gitlab # [ 1.948291] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.950652] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.952654] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.953676] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.954705] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.955661] systemd[1]: Reached target Socket Units. gitlab # [ 1.957848] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.959931] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.962165] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 1.973600] systemd[1]: Starting Journal Service... gitlab # [ 1.977602] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.978647] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.982815] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.984756] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.998474] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.000310] systemd-journald[92]: Collecting audit messages is disabled. gitlab # [ 2.003901] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.005348] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.011720] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.027569] netfs: FS-Cache loaded gitlab # [ 2.032385] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.034758] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.036446] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.039937] 9pnet: Installing 9P2000 support gitlab # [ 2.040355] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.042417] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.044412] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.047424] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.052583] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.054391] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.056379] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.057653] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.058872] systemd[1]: Reached target Local File Systems. gitlab # [ 1.912683] systemd-modules-load[93]: Using 4 probe threads gitlab # [ 2.060790] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.914740] systemd-modules-load[93]: Inserted module 'virtio[ 2.062056] systemd[1]: Started Journal Service. gitlab # _balloon' gitlab # [ 1.917159] systemd-modules-load[93]: Inserted module 'virtio_gpu' gitlab # [ 1.918300] systemd-modules-load[93]: Inserted module 'dm_mod' gitlab # [ 1.927853] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.940611] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.942695] systemd-udevd[104]: Using default interface naming scheme 'v261'. gitlab # [ 1.953090] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.972531] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 1.973411] systemd[1]: Reached target System Initialization. gitlab # [ 1.974229] systemd[1]: Reached target Basic System. gitlab # [ 2.194677] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.199062] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.200165] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.201861] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.202736] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.203647] SCSI subsystem initialized gitlab # [ 2.205233] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.206408] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.209126] usb usb1: Product: UHCI Host Controller gitlab # [ 2.210188] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.211752] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.212759] hub 1-0:1.0: USB hub found gitlab # [ 2.216041] hub 1-0:1.0: 2 ports detected gitlab # [ 2.217795] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.228634] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.231159] virtio_blk virtio5: [vdb] 11843008 512-byte logical blocks (6.06 GB/5.65 GiB) gitlab # [ 2.236487] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.239787] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.240476] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.247660] scsi host0: ata_piix gitlab # [ 2.275126] scsi host1: ata_piix gitlab # [ 2.276273] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.277672] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.138481] (udev-worker)[134]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.139933] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.144886] (udev-worker)[141]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.149140] (udev-worker)[141]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.158068] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.160260] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.161244] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.162138] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.163401] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.165044] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.186903] systemd-fsck[160]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.190871] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.435988] ata2: found unknown device (class 0) gitlab # [ 2.438854] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.442953] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.443820] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.470398] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.495761] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.527225] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.608464] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.611563] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.614279] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.615971] usb 1-1: Manufacturer: QEMU gitlab # [ 2.617553] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.647740] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.660909] usbcore: registered new interface driver usbhid gitlab # [ 2.661797] usbhid: USB HID core driver gitlab # [ 2.669892] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.674754] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.839968] systemd[1]: Mounting /sysroot... gitlab # [ 3.123580] EXT4-fs (vda): mounted filesystem ba7c3a22-777e-4c61-ba24-2f33c4003f0f r/w with ordered data mode. Quota mode: none. gitlab # [ 2.982278] systemd[1]: Mounted /sysroot. gitlab # [ 2.983034] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.985924] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.987750] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.989749] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.993751] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.006354] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.007354] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.017136] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.025181] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.171433] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.171548] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.029547] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.031681] systemd-sysroot-fstab-check[181]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.034642] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.035524] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.036319] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.040208] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.041303] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.044192] systemd[1]: Reload requested from client PID 181 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.045760] systemd[1]: Reloading... gitlab # [ 3.115933] systemd[1]: Reloading finished in 70 ms. gitlab # [ 3.138860] systemd-sysroot-fstab-check[181]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.141903] systemd-sysroot-fstab-check[181]: Requesting swap.target/start/replace... gitlab # [ 3.144149] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.148869] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.150661] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.152337] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.166211] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.168109] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.840275] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.867171] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.868405] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.870090] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.871502] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.889141] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.891047] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.899605] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.900573] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.902157] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.921273] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.922227] systemd[1]: Stopped target Basic System. gitlab # [ 3.923028] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.923836] systemd[1]: Stopped target Path Units. gitlab # [ 3.924619] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.925694] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.926773] systemd[1]: Stopped target Slice Units. gitlab # [ 3.929793] systemd[1]: Stopped target Socket Units. gitlab # [ 3.930586] systemd[1]: Stopped target System Initialization. gitlab # [ 3.931468] systemd[1]: Stopped target Swaps. gitlab # [ 3.932171] systemd[1]: Stopped target Timer Units. gitlab # [ 3.932938] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.933978] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.934983] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.936126] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.936854] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 3.938380] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.939320] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.940272] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.941039] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.942089] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.942887] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.943962] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.945066] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.946095] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.947048] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.947883] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.948876] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.949944] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.950878] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.951937] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.952950] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.953740] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.954750] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.955702] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.956714] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.957604] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.958553] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.959548] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.960533] systemd[1]: Closed udev Control Socket. gitlab # [ 3.961274] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.962115] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.963291] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.964187] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.965305] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.966300] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.967279] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 3.968191] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 3.969260] systemd[1]: Finished Cleanup udev Database. gitlab # [ 3.970046] systemd[1]: Reached target Switch Root. gitlab # [ 3.970825] systemd[1]: Starting NixOS Activation... gitlab # [ 4.040325] initrd-nixos-activation-start[228]: booting system configuration /nix/store/b6wl5763gmsvvwqv0qnqjb9nag9ipsgs-nixos-system-gitlab-test gitlab # [ 4.066311] initrd-nixos-activation-start[228]: running activation script... gitlab # [ 4.235265] initrd-nixos-activation-start[252]: setting up /etc... gitlab # [ 4.315358] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.316525] systemd[1]: Finished NixOS Activation. gitlab # [ 4.319687] systemd[1]: Starting Switch Root... gitlab # [ 4.338620] systemd[1]: Switching root. gitlab # [ 4.548943] systemd-journald[92]: Received SIGTERM from PID 1 (systemd). gitlab # [ 4.677288] NET: Registered PF_VSOCK protocol family gitlab # [ 5.057583] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.067778] systemd[1]: Detected virtualization kvm. gitlab # [ 5.069632] systemd[1]: Detected architecture x86-64. gitlab # [ 5.071620] systemd[1]: Detected first boot. gitlab # [ 5.074088] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.214305] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.274672] systemd[1]: Applying preset policy. gitlab # [ 5.421717] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.550623] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.551924] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.554669] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.556933] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.558307] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.559957] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.560922] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.562158] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.563273] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.564326] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.565930] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.566885] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.567685] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.568541] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.569447] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.570439] systemd[1]: Reached target Path Units. gitlab # [ 5.571210] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.572091] systemd[1]: Reached target Slice Units. gitlab # [ 5.572854] systemd[1]: Reached target Swaps. gitlab # [ 5.574458] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.576483] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.577880] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.579454] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.580422] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.582966] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.584320] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.585995] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.587192] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.588330] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.589670] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.592330] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.593437] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.594377] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.597330] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.599276] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.601305] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.603483] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.607615] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.608631] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 5.612567] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 5.626106] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.629738] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.633677] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 5.640179] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.642431] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.655646] systemd[1]: Starting Journal Service... gitlab # [ 5.657743] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.660657] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 5.666712] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 5.668660] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.673983] systemd-journald[322]: Collecting audit messages is enabled. gitlab # [ 5.675690] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 5.681203] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 5.683428] systemd[1]: Mounted Huge Pages File System. gitlab # [ 5.685667] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 5.687613] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 5.688611] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 5.689979] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 5.691348] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 5.546633] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 5.547928] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.694659] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 5.696171] systemd[1]: Started Journal Service. gitlab # [ 5.697205] loop: module loaded gitlab # [ 5.551652] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 5.699646] EXT4-fs (vda): re-mounted ba7c3a22-777e-4c61-ba24-2f33c4003f0f. gitlab # [ 5.553597] systemd-modules-load[323]: Using 3 probe threads gitlab # [ 5.555228] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.556344] systemd-modules-load[323]: Inserted module 'loop' gitlab # [ 5.559649] systemd-modules-load[323]: Inserted module 'tls' gitlab # [ 5.560667] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.564074] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.565329] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.570128] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 5.574711] systemd-oomd[324]: No swap; memory pressure usage will be degraded gitlab # [ 5.579450] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 5.580427] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.582489] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.583842] systemd[1]: Starting Firewall... gitlab # [ 5.585098] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.586140] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.587933] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 5.600093] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 5.601047] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.603979] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 5.754399] systemd-journald[322]: Received client request to flush runtime journal. gitlab # [ 5.647144] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 5.649661] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.650699] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.652074] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.652909] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.653808] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.654775] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.655982] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 5.657313] systemd-udevd[357]: Using default interface naming scheme 'v261'. gitlab # [ 5.681406] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 5.684524] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 5.755887] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 5.775306] (udev-worker)[431]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 5.778801] (udev-worker)[428]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 5.780916] (udev-worker)[428]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 5.784873] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 5.793657] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 5.795646] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.797249] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.801545] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.803211] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.804889] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.806199] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 5.985105] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 5.854035] systemd[1]: Finished Firewall. gitlab # [ 6.001771] ACPI: button: Power Button [PWRF] gitlab # [ 6.022060] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.029036] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.029048] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.029112] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.029174] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.033181] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.033279] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.036979] Console: switching to colour dummy device 80x25 gitlab # [ 6.040279] FDC 0 is a S82078B gitlab # [ 6.042255] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.042257] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.044052] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.044646] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.045665] rtc_cmos 00:05: setting system clock to 2026-09-06T16:05:26 UTC (1788710726) gitlab # [ 6.045673] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.051105] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.051731] systemd-journald[322]: Time jumped backwards, rotating. gitlab # [ 6.057171] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.058851] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.061886] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.078224] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.080188] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 gitlab # [ 6.090585] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 gitlab # [ 5.974184] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.149550] erofs (device vdb): failed to read inode meta block (nid: 129014395): -4 gitlab # [ 6.004770] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.005965] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.029037] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.035066] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.037224] s[ 6.183838] ppdev: user-space parallel port driver gitlab # ystemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.038828] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.206102] kvm_amd: TSC scaling supported gitlab # [ 6.206644] kvm_amd: Nested Virtualization enabled gitlab # [ 6.207174] kvm_amd: Nested Paging enabled gitlab # [ 6.207659] kvm_amd: LBR virtualization supported gitlab # [ 6.208174] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.208738] kvm_amd: Virtual GIF supported gitlab # [ 6.209203] kvm_amd: Virtual NMI enabled gitlab # [ 6.243051] EDAC MC: Ver: 3.0.0 gitlab # [ 6.104976] systemd-vconsole-setup[497]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.106992] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.406591] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.425571] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.426595] systemd[1]: Reached target Local File Systems. gitlab # [ 6.427604] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.428980] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.430376] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.431597] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.433313] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.440106] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.470660] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.474022] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.476470] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.496408] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.498413] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.499798] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.517773] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.520955] systemd[1]: Starting Update is Completed... gitlab # [ 6.545574] systemd[1]: Finished Update is Completed. gitlab # [ 6.574789] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.576602] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 6.825790] systemd[1]: Finished register-nix-paths.service. gitlab # [ 6.827060] systemd[1]: Reached target System Initialization. gitlab # [ 6.828288] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 6.829533] systemd[1]: Started logrotate.timer. gitlab # [ 6.830342] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 6.831317] systemd[1]: Reached target Timer Units. gitlab # [ 6.832099] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 6.833061] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 6.835120] systemd[1]: Starting nixbot CI service socket... gitlab # [ 6.836192] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 6.838716] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 6.839710] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 6.840644] systemd[1]: Reached target Socket Units. gitlab # [ 6.876303] dbus-broker-launch[579]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 6.878327] dbus-broker-launch[579]: NSS returned no entry for 'systemd-timesync' gitlab # [ 6.879979] dbus-broker-launch[579]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 6.883262] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 6.890538] dbus-broker-launch[579]: Ready gitlab # [ 6.891951] systemd[1]: Reached target Basic System. gitlab # [ 6.893579] systemd[1]: Started backdoor.service. gitlab # [ 6.895855] systemd[1]: Starting gitlab-config.service... gitlab # [ 6.905775] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 6.913059] systemd[1]: Starting Logrotate configuration check... gitlab # [ 6.914635] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 6.917681] systemd[1]: Starting Post-Boot Actions... gitlab # [ 6.921544] logrotate[584]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # connecting to host... gitlab # [ 6.924626] logrotate[584]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 6.926391] logrotate[584]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 6.927853] systemd[1]: Started Reset console on configuration changes. gitlab # [ 6.929313] logrotate[584]: Reading state from file: /var/lib/logrotate.status gitlab # [ 6.930781] logrotate[584]: state file /var/lib/logrotate.status does not exist gitlab # [ 6.932224] logrotate[584]: Allocating hash table for state file, size 64 entries gitlab # [ 6.933771] logrotate[584]: Handling 4 logs gitlab # [ 6.934641] logrotate[584]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 6.937407] logrotate[584]: considering log /var/log/btmp gitlab # [ 6.938443] logrotate[584]: Creating new state gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 6.939374] logrotate[584]: Now: 2026-09-06 16:05 gitlab # [ 6.940465] logrotate[584]: Last rotated at 2026-09-06 16:00 gitlab: connected to guest root shell gitlab # [ 6.941596] logrotate[584]: log does not need rotating (log has already been rotated) gitlab: (connecting took 7.54 seconds) gitlab # [ 6.942958] logrotate[584]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab: (finished: waiting for the VM to finish booting, in 7.54 seconds) gitlab # [ 6.945606] logrotate[584]: considering log /var/log/wtmp gitlab # [ 6.946608] logrotate[584]: Creating new state gitlab # [ 6.947438] logrotate[584]: Now: 2026-09-06 16:05 gitlab # [ 6.948416] logrotate[584]: Last rotated at 2026-09-06 16:00 gitlab # [ 6.949455] logrotate[584]: log does not need rotating (log has already been rotated) gitlab # [ 6.951310] logrotate[584]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 6.953236] logrotate[584]: switching euid from 0 to 165 and egid from 0 to 165 (pid 584) gitlab # [ 6.954700] logrotate[584]: considering log /var/gitlab/state/log/*.log gitlab # [ 6.955712] logrotate[584]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 6.956884] logrotate[584]: Creating new state gitlab # [ 6.957562] logrotate[584]: switching euid from 165 to 0 and egid from 165 to 0 (pid 584) gitlab # [ 6.958722] logrotate[584]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 6.960329] logrotate[584]: switching euid from 0 to 60 and egid from 0 to 60 (pid 584) gitlab # [ 6.961476] logrotate[584]: considering log /var/log/nginx/*.log gitlab # [ 6.962697] logrotate[584]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 6.963804] logrotate[584]: Creating new state gitlab # [ 6.964506] logrotate[584]: not running postrotate script, since no logs were rotated gitlab # [ 6.965619] logrotate[584]: switching euid from 60 to 0 and egid from 60 to 0 (pid 584) gitlab # [ 6.966749] systemd[1]: Starting resolvconf update... gitlab # [ 6.967736] systemd[1]: Finished Logrotate configuration check. gitlab # [ 6.968875] nsncd[585]: Sep 06 16:05:27.559 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 6.971122] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 6.971714] systemd[1]: Finished Post-Boot Actions. gitlab # [ 6.972934] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 6.973427] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 6.975046] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 6.975951] systemd[1]: Starting User Login Management... gitlab # [ 7.003240] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.004828] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.005231] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.006063] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.006502] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.007181] systemd-logind[631]: New seat seat0. gitlab # [ 7.010975] systemd-logind[631]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.012086] systemd-logind[631]: Watching system buttons on /dev/input/event4 (QEMU Virtio Keyboard) gitlab # [ 7.013128] systemd-logind[631]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.014159] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.014390] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.014856] systemd[1]: nscd.service: Consumed 19ms CPU time over 96ms wall clock time, 7.1M memory peak, 5.5M read from disk. gitlab # [ 7.015623] systemd[1]: Started User Login Management. gitlab # [ 7.021836] systemd[1]: Starting linger-users.service... gitlab # [ 7.022157] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.037533] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.038827] nsncd[675]: Sep 06 16:05:27.638 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.040808] systemd[1]: Finished linger-users.service. gitlab # [ 7.041630] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.042569] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.043469] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.045480] systemd[1]: Finished resolvconf update. gitlab # [ 7.046515] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.048473] systemd[1]: Starting DHCP Client... gitlab # [ 7.049821] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.052281] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.078343] network-addresses-eth1-start[702]: adding address 192.168.1.1/24... done gitlab # [ 7.081891] network-addresses-eth1-start[702]: adding address 2001:db8:1::1/64... done gitlab # [ 7.088523] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.110829] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.112306] systemd[1]: Reached target Network. gitlab # [ 7.115990] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.117654] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.119695] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.125778] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.127484] dhcpcd[712]: dhcpcd-10.3.2 starting gitlab # [ 7.129637] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.130139] dhcpcd[771]: dev: loaded udev gitlab # [ 7.281522] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.282042] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.148205] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.152103] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.158538] systemd[1]: Started Getty on tty1. gitlab # [ 7.161353] systemd[1]: Reached target Login Prompts. gitlab # [ 7.370715] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.375942] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.376831] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.377958] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.379084] cfg80211: failed to load regulatory.db gitlab # [ 7.245676] nginx-pre-start[792]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.246324] nginx-pre-start[792]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.251237] systemd[1]: Started Nginx Web Server. gitlab # [ 7.402257] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.257087] dhcpcd[771]: eth0: waiting for carrier gitlab # [ 7.257798] dhcpcd[771]: eth0: carrier acquired gitlab # [ 7.260943] dhcpcd[771]: DUID 00:01:00:01:32:30:4b:c7:52:54:00:12:34:56 gitlab # [ 7.261223] dhcpcd[771]: eth0: IAID 00:12:34:56 gitlab # [ 7.261827] dhcpcd[771]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.269573] redis-server[791]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.271813] redis-server[791]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.273038] redis-server[791]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=791, just started gitlab # [ 7.274677] redis-server[791]: Configuration loaded gitlab # [ 7.275650] redis[791]: monotonic clock: POSIX clock_gettime gitlab # [ 7.277340] redis[791]: Running mode=standalone, port=0. gitlab # [ 7.278866] redis[791]: Server initialized gitlab # [ 7.280042] redis[791]: Ready to accept connections unix gitlab # [ 7.281162] redis[791]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.283204] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.292495] postgresql-pre-start[796]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.293081] postgresql-pre-start[796]: This user must also own the server process. gitlab # [ 7.296038] postgresql-pre-start[796]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.296856] postgresql-pre-start[796]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.297379] postgresql-pre-start[796]: The default text search configuration will be set to "english". gitlab # [ 7.297887] postgresql-pre-start[796]: Data page checksums are enabled. gitlab # [ 7.298625] postgresql-pre-start[796]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.299429] postgresql-pre-start[796]: creating subdirectories ... ok gitlab # [ 7.300067] postgresql-pre-start[796]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.332548] postgresql-pre-start[796]: selecting default "max_connections" ... 100 gitlab # [ 7.371087] postgresql-pre-start[796]: selecting default "shared_buffers" ... 128MB gitlab # [ 7.582742] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 7.730497] postgresql-pre-start[796]: selecting default time zone ... UTC gitlab # [ 7.731959] postgresql-pre-start[796]: creating configuration files ... ok gitlab # [ 7.767363] gitlab-workhorse[827]: time="2026-09-06T16:05:28Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 7.767551] gitlab-workhorse[827]: time="2026-09-06T16:05:28Z" level=info msg="Using redis/go-redis" gitlab # [ 7.768041] gitlab-workhorse[827]: time="2026-09-06T16:05:28Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 7.903886] postgresql-pre-start[796]: running bootstrap script ... ok gitlab # [ 8.311163] postgresql-pre-start[796]: performing post-bootstrap initialization ... ok gitlab # [ 8.566297] postgresql-pre-start[796]: syncing data to disk ... ok gitlab # [ 8.566489] postgresql-pre-start[796]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 8.566930] postgresql-pre-start[796]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 8.567544] postgresql-pre-start[796]: Success. You can now start the database server using: gitlab # [ 8.568020] postgresql-pre-start[796]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 8.676296] postgres[852]: [852] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 8.678143] postgres[852]: [852] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 8.678325] postgres[852]: [852] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 8.683709] postgres[852]: [852] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 8.695178] postgres[861]: [861] LOG: database system was shut down at 2026-09-06 16:05:28 GMT gitlab # [ 8.702100] dhcpcd[771]: eth0: soliciting a DHCP lease gitlab # [ 8.706554] postgres[852]: [852] LOG: database system is ready to accept connections gitlab # [ 8.854953] NET: Registered PF_PACKET protocol family gitlab # [ 8.715126] systemd[1]: Started PostgreSQL Server. gitlab # [ 8.715476] dhcpcd[771]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 8.716715] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 8.721161] dhcpcd[771]: eth0: probing address 10.0.2.15/24 gitlab # [ 8.823743] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[866]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 8.824839] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[866]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 8.855545] systemd[1]: Finished gitlab-config.service. gitlab # [ 8.858865] postgresql-setup-start[879]: CREATE DATABASE gitlab # [ 8.859236] systemd[1]: Started gitaly.service. gitlab # [ 8.877178] postgresql-setup-start[897]: CREATE ROLE gitlab # [ 8.886186] postgresql-setup-start[900]: ALTER DATABASE gitlab # [ 8.905827] postgresql-setup-start[905]: CREATE ROLE gitlab # [ 8.908289] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 8.909117] systemd[1]: Reached target PostgreSQL. gitlab # [ 8.910260] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 8.911492] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.012691] gitlab-postgresql-start[915]: CREATE DATABASE gitlab # [ 9.046888] gitlab-postgresql-start[930]: CREATE EXTENSION gitlab # [ 9.128942] gitlab-postgresql-start[936]: CREATE EXTENSION gitlab # [ 9.130693] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 9.307370] gitaly[893]: time="2026-09-06T16:05:29.907Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=893 gitlab # [ 9.307773] gitaly[893]: time="2026-09-06T16:05:29.908Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=893 gitlab # [ 9.308447] gitaly[893]: time="2026-09-06T16:05:29.908Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=893 gitlab # [ 9.309083] gitaly[893]: time="2026-09-06T16:05:29.908Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=893 gitlab # [ 9.309560] gitaly[893]: time="2026-09-06T16:05:29.908Z" level=info msg="Starting Gitaly" pid=893 version= gitlab # [ 9.492413] gitaly[893]: time="2026-09-06T16:05:30.093Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=184 pid=893 gitlab # [ 9.494325] gitaly[893]: time="2026-09-06T16:05:30.093Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=893 gitlab # [ 9.496104] gitaly[893]: time="2026-09-06T16:05:30.093Z" level=info msg="finished initializing command factory" duration_ms=0 pid=893 gitlab # [ 9.497852] gitaly[893]: time="2026-09-06T16:05:30.093Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-893/git-exec-1390139134.d/git pid=893 gitlab # [ 9.499890] gitaly[893]: time="2026-09-06T16:05:30.097Z" level=info msg="finished detecting git version" duration_ms=3 pid=893 gitlab # [ 9.501502] gitaly[893]: time="2026-09-06T16:05:30.097Z" level=info msg="using Git version" pid=893 version=2.53.g gitlab # [ 9.503048] gitaly[893]: time="2026-09-06T16:05:30.098Z" level=info msg="clearing disk cache object folder" pid=893 storage=default gitlab # [ 9.504724] gitaly[893]: time="2026-09-06T16:05:30.098Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache909533163 pid=893 storage=default gitlab # [ 9.507394] gitaly[893]: time="2026-09-06T16:05:30.098Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache909533163 pid=893 storage=default gitlab # [ 9.510337] gitaly[893]: time="2026-09-06T16:05:30.098Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=893 gitlab # [ 9.513059] gitaly[893]: time="2026-09-06T16:05:30.098Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=893 gitlab # [ 9.515739] gitaly[893]: time="2026-09-06T16:05:30.098Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache909533163 clear_duration_ms=0 pid=893 storage=default gitlab # [ 9.540332] gitaly[893]: time="2026-09-06T16:05:30.141Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=893 gitlab # [ 9.542826] gitaly[893]: time="2026-09-06T16:05:30.141Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=893 schema=unix gitlab # [ 9.544931] gitaly[893]: time="2026-09-06T16:05:30.141Z" level=info msg="listening at address" address=/run/gitaly/gitaly-893/sock.d/intern pid=893 schema=unix gitlab # [ 9.547041] gitaly[893]: time="2026-09-06T16:05:30.141Z" level=info msg="finished tempdir cleaner walk" pid=893 storage=default time_ms=0 gitlab # [ 9.547488] gitaly[893]: time="2026-09-06T16:05:30.141Z" level=info msg="counting repositories" pid=893 gitlab # [ 9.548060] gitaly[893]: time="2026-09-06T16:05:30.141Z" level=info msg="starting to count repositories" pid=893 storage_path=/var/gitlab/state/repositories gitlab # [ 9.548624] gitaly[893]: time="2026-09-06T16:05:30.141Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=893 storage_path=/var/gitlab/state/repositories gitlab # [ 9.549224] gitaly[893]: time="2026-09-06T16:05:30.141Z" level=info msg="completed counting all repositories" counting_duration=0 pid=893 gitlab # [ 9.550231] gitaly[893]: time="2026-09-06T16:05:30.144Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=893 gitlab # [ 9.551232] gitaly[893]: time="2026-09-06T16:05:30.144Z" level=info msg="Started Gitaly" duration_ms=236 pid=893 gitlab # [ 9.551922] gitaly[893]: time="2026-09-06T16:05:30.144Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=893 gitlab # [ 9.552848] gitaly[893]: time="2026-09-06T16:05:30.144Z" level=info msg="maintenance: daily scheduled" pid=893 scheduled="2026-09-07 12:00:00 +0000 UTC" gitlab # [ 9.566671] dhcpcd[771]: eth0: soliciting an IPv6 router gitlab # [ 9.567871] dhcpcd[771]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.568941] dhcpcd[771]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.569587] dhcpcd[771]: eth0: adding route to fec0::/64 gitlab # [ 9.570115] dhcpcd[771]: eth0: adding default route via fe80::2 gitlab # [ 10.044793] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[910]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.045492] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[910]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.069808] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[910]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.702316] gitaly[893]: time="2026-09-06T16:05:32.302Z" level=info msg="License database preloaded" duration_ms=2394 pid=893 gitlab # [ 14.554665] dhcpcd[771]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.555473] dhcpcd[771]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.556766] dhcpcd[771]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.645534] systemd[1]: Started DHCP Client. gitlab # [ 14.646772] systemd[1]: Reached target Network is Online. gitlab # [ 42.503264] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[910]: W, [2026-09-06T16:06:03.103608 #910] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 44.674494] postgres[1337]: [1337] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 44.674939] postgres[1337]: [1337] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 44.675687] postgres[1337]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 44.676712] postgres[1337]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 44.677547] postgres[1337]: attidentity AS identity, gitlab # [ 44.678447] postgres[1337]: attgenerated as attgenerated gitlab # [ 44.679200] postgres[1337]: FROM pg_attribute a gitlab # [ 44.679851] postgres[1337]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 44.681719] postgres[1337]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 44.682955] postgres[1337]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 44.683589] postgres[1337]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 44.686810] postgres[1337]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 44.687613] postgres[1337]: ORDER BY a.attnum gitlab # [ 44.688572] postgres[1337]: gitlab # [ 47.809165] gitlab-workhorse[827]: time="2026-09-06T16:06:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 48.637409] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1379]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 48.637700] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1379]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 57.819097] gitlab-workhorse[827]: time="2026-09-06T16:06:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 63.808130] postgres[1445]: [1445] ERROR: relation "table_without_models" does not exist gitlab # [ 63.811062] postgres[1445]: [1445] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 67.829312] gitlab-workhorse[827]: time="2026-09-06T16:06:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 77.830923] gitlab-workhorse[827]: time="2026-09-06T16:06:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 81.535149] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 81.545763] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 81.547201] systemd[1]: Starting gitlab.service... gitlab # [ 82.161861] bundle[1666]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 82.164651] bundle[1666]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 82.166493] bundle[1666]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 85.834668] sidekiq-cluster[1665]: {"severity":"INFO","time":"2026-09-06T16:06:46.403Z","message":"Starting cluster with 1 processes"} gitlab # [ 85.837332] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.409Z","pid":1666,"message":"* Enabling systemd notification integration"} gitlab # [ 85.838837] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.435Z","pid":1666,"message":"Puma starting in cluster mode..."} gitlab # [ 85.849183] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.435Z","pid":1666,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 85.854723] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.436Z","pid":1666,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 85.859168] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.436Z","pid":1666,"message":"* Min threads: 0"} gitlab # [ 85.863089] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.436Z","pid":1666,"message":"* Max threads: 4"} gitlab # [ 85.867169] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.436Z","pid":1666,"message":"* Environment: production"} gitlab # [ 85.871842] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.436Z","pid":1666,"message":"* Master PID: 1666"} gitlab # [ 85.873108] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.436Z","pid":1666,"message":"* Workers: 2"} gitlab # [ 85.876610] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.436Z","pid":1666,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 85.881633] bundle[1666]: {"timestamp":"2026-09-06T16:06:46.436Z","pid":1666,"message":"* Preloading application"} gitlab # [ 87.851796] gitlab-workhorse[827]: time="2026-09-06T16:06:48Z" level=error correlation_id= duration_ms=2 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 97.860553] gitlab-workhorse[827]: time="2026-09-06T16:06:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 107.861284] gitlab-workhorse[827]: time="2026-09-06T16:07:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 112.118904] bundle[1666]: W, [2026-09-06T16:07:12.719222 #1666] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 115.348046] sidekiq-cluster[1706]: 2026-09-06T16:07:15.948Z pid=1706 tid=wea INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1706, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432136192} gitlab # [ 115.552389] sidekiq-cluster[1706]: W, [2026-09-06T16:07:16.152878 #1706] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 115.597551] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:16.198Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 117.753368] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.353Z","message":"GitLab reliable fetch activated!"} gitlab # [ 117.753597] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.353Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 117.778600] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.379Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 117.781311] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.382Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 117.783995] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.384Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 117.786887] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.387Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 117.789400] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.390Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 117.791688] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.392Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 117.793111] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.393Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 117.794636] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.395Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 117.796050] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.396Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 117.797554] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.398Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 117.799079] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.399Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 117.801185] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.401Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 117.802893] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.403Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 117.804476] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.405Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 117.806613] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.407Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 117.809882] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.410Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 117.812817] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.413Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 117.815034] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.415Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 117.817542] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.418Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 117.820569] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.421Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 117.822808] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.423Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 117.824596] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.425Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 117.826275] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.427Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 117.828302] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.429Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 117.830130] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.430Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 117.832632] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.433Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 117.834772] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.435Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 117.836923] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.437Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 117.839095] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.439Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 117.841241] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.441Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 117.842888] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.443Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 117.845220] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.445Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 117.847762] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.448Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 117.849449] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.450Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 117.851491] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.452Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 117.853457] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.454Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 117.855371] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.456Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 117.857726] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.458Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 117.859531] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.460Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 117.861117] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.461Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 117.863131] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.463Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 117.865135] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.465Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 117.867102] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.467Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 117.869105] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.469Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 117.870651] gitlab-workhorse[827]: time="2026-09-06T16:07:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 117.871722] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.472Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 117.873348] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.474Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 117.874793] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.475Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 117.876796] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.477Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 117.878768] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.479Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 117.880796] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.481Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 117.882716] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.483Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 117.885080] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.485Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 117.894199] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.494Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 117.900143] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.500Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 117.905353] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.506Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 117.908814] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.509Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 117.913176] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.513Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 117.920307] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.521Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 117.925653] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.526Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 117.931241] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.531Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 117.936798] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.537Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 117.941754] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.542Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 117.948307] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.549Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 117.953243] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.553Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 117.957769] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.558Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 117.962259] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.563Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 117.965791] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.566Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 118.013736] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.614Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 118.017977] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.618Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 118.021641] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.622Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 118.025308] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.626Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 118.029116] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.629Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 118.033440] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.634Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 118.037083] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.637Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 118.041254] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.642Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 118.044977] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.645Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 118.049136] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.649Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 118.053106] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.653Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 118.057038] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.657Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 118.060414] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.661Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 118.064449] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.665Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 118.068665] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.669Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 118.073062] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.673Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 118.077139] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.677Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 118.081222] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.681Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 118.085471] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.686Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 118.089557] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.690Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 118.093642] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.694Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 118.097304] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.698Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 118.102812] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.703Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 118.107334] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.708Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 118.112449] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.713Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 118.117311] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.717Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 118.122191] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.722Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 118.127429] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.728Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 118.134099] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.734Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 118.137910] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.738Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 118.142290] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.742Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 118.157052] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.757Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 118.161160] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.761Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 118.165048] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.765Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 118.170210] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.770Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 118.174474] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.775Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 118.178867] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.779Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 118.183302] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.784Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 118.187406] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.788Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 118.192044] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.792Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 118.199870] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.800Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 118.204867] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:18.805Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 127.871783] gitlab-workhorse[827]: time="2026-09-06T16:07:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 137.880978] gitlab-workhorse[827]: time="2026-09-06T16:07:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 141.177340] postgres[2217]: [2217] ERROR: relation "table_without_models" does not exist gitlab # [ 141.178143] postgres[2217]: [2217] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 143.079679] postgres[2237]: [2237] ERROR: relation "table_without_models" does not exist gitlab # [ 143.081376] postgres[2237]: [2237] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 143.712867] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:44.313Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 143.718424] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:44.319Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 143.720274] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:44.319Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 143.721945] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:44.319Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 143.723830] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:44.321Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 145.240861] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:45.840Z","message":"Cleaning working queues"} gitlab # [ 145.247476] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:45.847Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"71df092fe214d0887516e047","created_at":"2026-09-06T16:06:41.157Z","trace_propagation_headers":{"sentry-trace":"668bc8cd1763473cae2ef8197c022233-ef20612201e84aca","baggage":"sentry-trace_id=668bc8cd1763473cae2ef8197c022233,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"954d26d17bda58aff217257197e3d4d8","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:06:41.171Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-71df092fe214d0887516e047: start","job_status":"start","queue_duration_s":64.676443,"scheduling_latency_s":64.676504} gitlab # [ 145.443097] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:07:46.043Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"71df092fe214d0887516e047","created_at":"2026-09-06T16:06:41.157Z","trace_propagation_headers":{"sentry-trace":"668bc8cd1763473cae2ef8197c022233-ef20612201e84aca","baggage":"sentry-trace_id=668bc8cd1763473cae2ef8197c022233,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"954d26d17bda58aff217257197e3d4d8","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:06:41.171Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-71df092fe214d0887516e047: done: 0.195886 sec","job_status":"done","queue_duration_s":64.676443,"scheduling_latency_s":64.676504,"redis_calls":31,"redis_duration_s":0.006558,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.004307,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001292,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.093422489,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.195886,"completed_at":"2026-09-06T16:07:46.043Z","load_balancing_strategy":"primary","db_duration_s":0.059849,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 146.014231] bundle[1666]: {"timestamp":"2026-09-06T16:07:46.614Z","pid":1666,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 146.014910] bundle[1666]: {"timestamp":"2026-09-06T16:07:46.614Z","pid":1666,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 146.016426] bundle[1666]: {"timestamp":"2026-09-06T16:07:46.614Z","pid":1666,"message":"! #\u003cThread:0x0000728fea16afc0 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 146.018070] bundle[1666]: {"timestamp":"2026-09-06T16:07:46.614Z","pid":1666,"message":"! #\u003cThread:0x0000728fea3aa010 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 146.019085] bundle[1666]: {"timestamp":"2026-09-06T16:07:46.614Z","pid":1666,"message":"Use Ctrl-C to stop"} gitlab # [ 146.020209] bundle[1666]: {"timestamp":"2026-09-06T16:07:46.614Z","pid":1666,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 146.426363] bundle[1666]: {"timestamp":"2026-09-06T16:07:47.026Z","pid":1666,"message":"- Worker 1 (PID: 2295) booted in 0.33s, phase: 0"} gitlab # [ 146.430459] bundle[1666]: {"timestamp":"2026-09-06T16:07:47.028Z","pid":1666,"message":"- Worker 0 (PID: 2291) booted in 0.35s, phase: 0"} gitlab # [ 146.434756] systemd[1]: Started gitlab.service. gitlab # [ 146.436547] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 146.439071] systemd[1]: Starting setup-gitlab.service... gitlab # [ 146.461344] setup-gitlab-start[2328]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 148.917265] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:07:49 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2429 gitlab # [ 148.922129] setup-gitlab-start[2328]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 148.924465] setup-gitlab-start[2328]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 148.935826] runuser[2351]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 149.315133] setup-gitlab-start[2352]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.319883] setup-gitlab-start[2352]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.324646] setup-gitlab-start[2352]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.950719] setup-gitlab-start[2352]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.953131] setup-gitlab-start[2352]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.955038] setup-gitlab-start[2352]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 162.339231] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:02.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b478d0ae769a0fd1253a00cf","created_at":"2026-09-06T16:08:02.900Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a33295450c416e0279cab1a9321ac117","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:02.938Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b478d0ae769a0fd1253a00cf: start","job_status":"start","queue_duration_s":0.001292,"scheduling_latency_s":0.001317} gitlab # [ 162.383551] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:02.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b478d0ae769a0fd1253a00cf","created_at":"2026-09-06T16:08:02.900Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a33295450c416e0279cab1a9321ac117","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:02.938Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b478d0ae769a0fd1253a00cf: done: 0.044627 sec","job_status":"done","queue_duration_s":0.001292,"scheduling_latency_s":0.001317,"redis_calls":13,"redis_duration_s":0.005763,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00116,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002127,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002476,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.013926630999999998,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.044627,"completed_at":"2026-09-06T16:08:02.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009322,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 162.427271] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:02.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b01e997d57b193a67a01ed97","created_at":"2026-09-06T16:08:02.947Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e0ba618aae4671abdbaa5850c3e37d3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:02.956Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b01e997d57b193a67a01ed97: start","job_status":"start","queue_duration_s":0.02838,"scheduling_latency_s":0.028394} gitlab # [ 162.440850] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:02.994Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 162.444391] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:02.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b01e997d57b193a67a01ed97","created_at":"2026-09-06T16:08:02.947Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e0ba618aae4671abdbaa5850c3e37d3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:02.956Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b01e997d57b193a67a01ed97: done: 0.014646 sec","job_status":"done","queue_duration_s":0.02838,"scheduling_latency_s":0.028394,"redis_calls":10,"redis_duration_s":0.001233,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199746,"duration_s":0.014646,"completed_at":"2026-09-06T16:08:02.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.481439] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a95d851e6e8b7fe30447a160","created_at":"2026-09-06T16:08:02.971Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7235e8d772d54afa4814d9d90e51023","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.007Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a95d851e6e8b7fe30447a160: start","job_status":"start","queue_duration_s":0.003214,"scheduling_latency_s":0.003228} gitlab # [ 162.494272] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.044Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a95d851e6e8b7fe30447a160","created_at":"2026-09-06T16:08:02.971Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7235e8d772d54afa4814d9d90e51023","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.007Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a95d851e6e8b7fe30447a160: done: 0.033069 sec","job_status":"done","queue_duration_s":0.003214,"scheduling_latency_s":0.003228,"redis_calls":12,"redis_duration_s":0.006932000000000001,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0014,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001361,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000824127,"duration_s":0.033069,"completed_at":"2026-09-06T16:08:03.044Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.532064] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.090Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0a626a278c82ceefe7ac29af","created_at":"2026-09-06T16:08:03.010Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"153f0a0a04c9ddf25d25403fef7bddb5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.025Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0a626a278c82ceefe7ac29af: start","job_status":"start","queue_duration_s":0.064877,"scheduling_latency_s":0.064921} gitlab # [ 162.644806] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0a626a278c82ceefe7ac29af","created_at":"2026-09-06T16:08:03.010Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"153f0a0a04c9ddf25d25403fef7bddb5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.025Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0a626a278c82ceefe7ac29af: done: 0.15493 sec","job_status":"done","queue_duration_s":0.064877,"scheduling_latency_s":0.064921,"redis_calls":14,"redis_duration_s":0.010398,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002048,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002972,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005378,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001556622,"exclusive_lock_hold_duration_s":0.105227341,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.15493,"completed_at":"2026-09-06T16:08:03.244Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.091541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.712758] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"de8370a4abbb1a3c03ebd863","created_at":"2026-09-06T16:08:03.032Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6aedb45ae63a9afd70c4a476999ece1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.043Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-de8370a4abbb1a3c03ebd863: start","job_status":"start","queue_duration_s":0.26974,"scheduling_latency_s":0.269767} gitlab # [ 162.783314] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.383Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 162.805190] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"de8370a4abbb1a3c03ebd863","created_at":"2026-09-06T16:08:03.032Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6aedb45ae63a9afd70c4a476999ece1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.043Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-de8370a4abbb1a3c03ebd863: done: 0.092187 sec","job_status":"done","queue_duration_s":0.26974,"scheduling_latency_s":0.269767,"redis_calls":10,"redis_duration_s":0.036676,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.025705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001572546,"duration_s":0.092187,"completed_at":"2026-09-06T16:08:03.405Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.867111] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dc87f77ff21ae378ffad94f9","created_at":"2026-09-06T16:08:03.092Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d5fb2edf4d9be681ae16d0e8539e3e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.104Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dc87f77ff21ae378ffad94f9: start","job_status":"start","queue_duration_s":0.362642,"scheduling_latency_s":0.362703} gitlab # [ 163.196101] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dc87f77ff21ae378ffad94f9","created_at":"2026-09-06T16:08:03.092Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d5fb2edf4d9be681ae16d0e8539e3e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.104Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dc87f77ff21ae378ffad94f9: done: 0.329131 sec","job_status":"done","queue_duration_s":0.362642,"scheduling_latency_s":0.362703,"redis_calls":14,"redis_duration_s":0.010831,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003661,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003648,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003522,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.161928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001516673,"exclusive_lock_hold_duration_s":0.254075867,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.329131,"completed_at":"2026-09-06T16:08:03.796Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.128802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.198464] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.797Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"091e05ca1b5fcaa5ed90716a","created_at":"2026-09-06T16:08:03.114Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"183c9f0e54c15470825bc189d12fcb76","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.121Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-091e05ca1b5fcaa5ed90716a: start","job_status":"start","queue_duration_s":0.676326,"scheduling_latency_s":0.676341} gitlab # [ 163.327276] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.927Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"091e05ca1b5fcaa5ed90716a","created_at":"2026-09-06T16:08:03.114Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"183c9f0e54c15470825bc189d12fcb76","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.121Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-091e05ca1b5fcaa5ed90716a: done: 0.12968 sec","job_status":"done","queue_duration_s":0.676326,"scheduling_latency_s":0.676341,"redis_calls":16,"redis_duration_s":0.0042699999999999995,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002617,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.092,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.092,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000513752,"duration_s":0.12968,"completed_at":"2026-09-06T16:08:03.927Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.095052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.330714] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.931Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b7fb297c62d4d66b0d065c44","created_at":"2026-09-06T16:08:03.125Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e328ae04edbbc3295bf623a2474963f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.142Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b7fb297c62d4d66b0d065c44: start","job_status":"start","queue_duration_s":0.788609,"scheduling_latency_s":0.788644} gitlab # [ 163.382770] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.982Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b7fb297c62d4d66b0d065c44","created_at":"2026-09-06T16:08:03.125Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e328ae04edbbc3295bf623a2474963f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.142Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b7fb297c62d4d66b0d065c44: done: 0.051239 sec","job_status":"done","queue_duration_s":0.788609,"scheduling_latency_s":0.788644,"redis_calls":12,"redis_duration_s":0.004184,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000588,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001944,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001652,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000838095,"exclusive_lock_hold_duration_s":0.016093666,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.051239,"completed_at":"2026-09-06T16:08:03.982Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01368,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.391550] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8a7c325e1a121a428098432a","created_at":"2026-09-06T16:08:03.147Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c83a943ef3326c064c853ad5476d8870","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:08:03.155Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8a7c325e1a121a428098432a: start","job_status":"start","queue_duration_s":0.829132,"scheduling_latency_s":0.82916} gitlab # [ 163.445345] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:03.997Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 163.446242] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8a7c325e1a121a428098432a","created_at":"2026-09-06T16:08:03.147Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c83a943ef3326c064c853ad5476d8870","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:08:03.155Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8a7c325e1a121a428098432a: done: 0.017685 sec","job_status":"done","queue_duration_s":0.829132,"scheduling_latency_s":0.82916,"redis_calls":10,"redis_duration_s":0.001658,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000262,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000244165,"duration_s":0.017685,"completed_at":"2026-09-06T16:08:04.001Z","load_balancing_strategy":"primary","db_duration_s":0.001139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.449568] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3fd4403cf3457d05642a9d53","created_at":"2026-09-06T16:08:03.158Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.161Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3fd4403cf3457d05642a9d53: start","job_status":"start","queue_duration_s":0.841274,"scheduling_latency_s":0.841291} gitlab # [ 163.469612] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3fd4403cf3457d05642a9d53","created_at":"2026-09-06T16:08:03.158Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.161Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3fd4403cf3457d05642a9d53: done: 0.067007 sec","job_status":"done","queue_duration_s":0.841274,"scheduling_latency_s":0.841291,"redis_calls":71,"redis_duration_s":0.011791,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001623,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000562,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.009117,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000360661,"duration_s":0.067007,"completed_at":"2026-09-06T16:08:04.069Z","load_balancing_strategy":"primary","db_duration_s":0.001551,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.521866] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fb01ccaf35909a847abaf173","created_at":"2026-09-06T16:08:03.164Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33dcbf944f9115d7fa46d824571f81dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.167Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fb01ccaf35909a847abaf173: start","job_status":"start","queue_duration_s":0.903492,"scheduling_latency_s":0.903512} gitlab # [ 163.522607] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.080Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 163.523068] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.084Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fb01ccaf35909a847abaf173","created_at":"2026-09-06T16:08:03.164Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33dcbf944f9115d7fa46d824571f81dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.167Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fb01ccaf35909a847abaf173: done: 0.013774 sec","job_status":"done","queue_duration_s":0.903492,"scheduling_latency_s":0.903512,"redis_calls":10,"redis_duration_s":0.0012699999999999999,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208685,"duration_s":0.013774,"completed_at":"2026-09-06T16:08:04.084Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001381,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.563560] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"2b958d0dc1b0dcea333c434e","created_at":"2026-09-06T16:08:03.170Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4c3653b16b18e33e9df61291eb8658f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.174Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-2b958d0dc1b0dcea333c434e: start","job_status":"start","queue_duration_s":0.911575,"scheduling_latency_s":0.91159} gitlab # [ 163.571630] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.105Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"2b958d0dc1b0dcea333c434e","created_at":"2026-09-06T16:08:03.170Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4c3653b16b18e33e9df61291eb8658f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.174Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-2b958d0dc1b0dcea333c434e: done: 0.020129 sec","job_status":"done","queue_duration_s":0.911575,"scheduling_latency_s":0.91159,"redis_calls":10,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000333562,"duration_s":0.020129,"completed_at":"2026-09-06T16:08:04.105Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.622872] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.107Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e7c90b8a0e6835363c8e6e24","created_at":"2026-09-06T16:08:03.181Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce2aeebf18788019380884d13271d25d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.184Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e7c90b8a0e6835363c8e6e24: start","job_status":"start","queue_duration_s":0.922475,"scheduling_latency_s":0.92249} gitlab # [ 163.741143] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e7c90b8a0e6835363c8e6e24","created_at":"2026-09-06T16:08:03.181Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce2aeebf18788019380884d13271d25d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.184Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e7c90b8a0e6835363c8e6e24: done: 0.234401 sec","job_status":"done","queue_duration_s":0.922475,"scheduling_latency_s":0.92249,"redis_calls":565,"redis_duration_s":0.063584,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.062254,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.187781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000237739,"duration_s":0.234401,"completed_at":"2026-09-06T16:08:04.341Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.742078] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.342Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"78f8c7360c72d6b4ffe62c33","created_at":"2026-09-06T16:08:03.196Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8678cc16c21573ff0072d0db1ab3cc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:08:03.203Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-78f8c7360c72d6b4ffe62c33: start","job_status":"start","queue_duration_s":1.138905,"scheduling_latency_s":1.138926} gitlab # [ 163.804109] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.364Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"78f8c7360c72d6b4ffe62c33","created_at":"2026-09-06T16:08:03.196Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8678cc16c21573ff0072d0db1ab3cc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:08:03.203Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-78f8c7360c72d6b4ffe62c33: done: 0.021649 sec","job_status":"done","queue_duration_s":1.138905,"scheduling_latency_s":1.138926,"redis_calls":10,"redis_duration_s":0.001495,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000269029,"duration_s":0.021649,"completed_at":"2026-09-06T16:08:04.364Z","load_balancing_strategy":"primary","db_duration_s":0.002948,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.846266] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"7df14ece5a995690e282a9ad","created_at":"2026-09-06T16:08:03.213Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e01e7c0d95fcb4e0a0327fa486de2d89","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.218Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-7df14ece5a995690e282a9ad: start","job_status":"start","queue_duration_s":1.147369,"scheduling_latency_s":1.147382} gitlab # [ 163.846950] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"7df14ece5a995690e282a9ad","created_at":"2026-09-06T16:08:03.213Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e01e7c0d95fcb4e0a0327fa486de2d89","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.218Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-7df14ece5a995690e282a9ad: done: 0.015952 sec","job_status":"done","queue_duration_s":1.147369,"scheduling_latency_s":1.147382,"redis_calls":12,"redis_duration_s":0.0024950000000000003,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000776,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00036513,"exclusive_lock_hold_duration_s":0.003539836,"duration_s":0.015952,"completed_at":"2026-09-06T16:08:04.381Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002129,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.884789] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"45854ed7e178741e8c0a22e7","created_at":"2026-09-06T16:08:03.220Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8dc6b87eaaf7c3ee00bc18960c03a29","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.226Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-45854ed7e178741e8c0a22e7: start","job_status":"start","queue_duration_s":1.156006,"scheduling_latency_s":1.156019} gitlab # [ 163.885706] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"45854ed7e178741e8c0a22e7","created_at":"2026-09-06T16:08:03.220Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8dc6b87eaaf7c3ee00bc18960c03a29","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.226Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-45854ed7e178741e8c0a22e7: done: 0.017644 sec","job_status":"done","queue_duration_s":1.156006,"scheduling_latency_s":1.156019,"redis_calls":12,"redis_duration_s":0.002532,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000397,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000396977,"exclusive_lock_hold_duration_s":0.002699226,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.017644,"completed_at":"2026-09-06T16:08:04.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.926821] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b6e83225097a5ac26070a3bf","created_at":"2026-09-06T16:08:03.234Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fee0b2a8663ab183f5de065aed6f517","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:08:03.242Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b6e83225097a5ac26070a3bf: start","job_status":"start","queue_duration_s":1.15881,"scheduling_latency_s":1.158834} gitlab # [ 163.927604] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.413Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 163.965314] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.419Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b6e83225097a5ac26070a3bf","created_at":"2026-09-06T16:08:03.234Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fee0b2a8663ab183f5de065aed6f517","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:08:03.242Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b6e83225097a5ac26070a3bf: done: 0.017866 sec","job_status":"done","queue_duration_s":1.15881,"scheduling_latency_s":1.158834,"redis_calls":10,"redis_duration_s":0.001843,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000333,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001093,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000302552,"duration_s":0.017866,"completed_at":"2026-09-06T16:08:04.419Z","load_balancing_strategy":"primary","db_duration_s":0.000419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.966421] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b006163e8fe8ad65ef2a6ec2","created_at":"2026-09-06T16:08:03.333Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3bc19abc4de279c19442784d5a285ba5","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.351Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b006163e8fe8ad65ef2a6ec2: start","job_status":"start","queue_duration_s":1.068912,"scheduling_latency_s":1.068927} gitlab # [ 164.010136] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.438Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b006163e8fe8ad65ef2a6ec2","created_at":"2026-09-06T16:08:03.333Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3bc19abc4de279c19442784d5a285ba5","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.351Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b006163e8fe8ad65ef2a6ec2: done: 0.01874 sec","job_status":"done","queue_duration_s":1.068912,"scheduling_latency_s":1.068927,"redis_calls":10,"redis_duration_s":0.001255,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000211759,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.01874,"completed_at":"2026-09-06T16:08:04.438Z","load_balancing_strategy":"primary","db_duration_s":0.005674,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.011050] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.439Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"dee1cec7916b97ff1bd9c3ce","created_at":"2026-09-06T16:08:03.384Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"2d86af116dd0b106aff2433f85659a4e","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.403Z","job_size_bytes":3,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-dee1cec7916b97ff1bd9c3ce: start","job_status":"start","queue_duration_s":1.036328,"scheduling_latency_s":1.036342} gitlab # [ 164.048368] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.452Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"dee1cec7916b97ff1bd9c3ce","created_at":"2026-09-06T16:08:03.384Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"2d86af116dd0b106aff2433f85659a4e","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.403Z","job_size_bytes":3,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-dee1cec7916b97ff1bd9c3ce: done: 0.012255 sec","job_status":"done","queue_duration_s":1.036328,"scheduling_latency_s":1.036342,"redis_calls":10,"redis_duration_s":0.002407,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000221,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001838,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000584153,"duration_s":0.012255,"completed_at":"2026-09-06T16:08:04.452Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000234,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.057269] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"865e066b25786e59585d914e","created_at":"2026-09-06T16:08:03.481Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f66445ba81ded690b43a48740aaeade2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.499Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-865e066b25786e59585d914e: start","job_status":"start","queue_duration_s":0.953569,"scheduling_latency_s":0.953583} gitlab # [ 164.103698] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.460Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 164.104459] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"865e066b25786e59585d914e","created_at":"2026-09-06T16:08:03.481Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f66445ba81ded690b43a48740aaeade2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.499Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-865e066b25786e59585d914e: done: 0.012231 sec","job_status":"done","queue_duration_s":0.953569,"scheduling_latency_s":0.953583,"redis_calls":10,"redis_duration_s":0.001263,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000232,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200863,"duration_s":0.012231,"completed_at":"2026-09-06T16:08:04.465Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.111610] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"52ed65064910c023e3643065","created_at":"2026-09-06T16:08:03.536Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7cfcbfe63bc4e048c1f987b4a520909b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.545Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-52ed65064910c023e3643065: start","job_status":"start","queue_duration_s":0.920922,"scheduling_latency_s":0.920939} gitlab # [ 164.155282] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"52ed65064910c023e3643065","created_at":"2026-09-06T16:08:03.536Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7cfcbfe63bc4e048c1f987b4a520909b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.545Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-52ed65064910c023e3643065: done: 0.015645 sec","job_status":"done","queue_duration_s":0.920922,"scheduling_latency_s":0.920939,"redis_calls":10,"redis_duration_s":0.00173,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000396,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000284114,"duration_s":0.015645,"completed_at":"2026-09-06T16:08:04.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.158950] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.483Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2b11b06223b99e057122a73a","created_at":"2026-09-06T16:08:03.552Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.561Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2b11b06223b99e057122a73a: start","job_status":"start","queue_duration_s":0.922059,"scheduling_latency_s":0.922081} gitlab # [ 164.196850] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2b11b06223b99e057122a73a","created_at":"2026-09-06T16:08:03.552Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.561Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2b11b06223b99e057122a73a: done: 0.033732 sec","job_status":"done","queue_duration_s":0.922059,"scheduling_latency_s":0.922081,"redis_calls":19,"redis_duration_s":0.003189,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000763,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000259,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001613,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000554,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000234108,"duration_s":0.033732,"completed_at":"2026-09-06T16:08:04.516Z","load_balancing_strategy":"primary","db_duration_s":0.002122,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.205756] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.518Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"6aa627df94a58271baa523c6","created_at":"2026-09-06T16:08:03.569Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c57d272cbbd1a5ddb496ba64166d67da","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.582Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-6aa627df94a58271baa523c6: start","job_status":"start","queue_duration_s":0.935381,"scheduling_latency_s":0.935421} gitlab # [ 164.242399] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.537Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"6aa627df94a58271baa523c6","created_at":"2026-09-06T16:08:03.569Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c57d272cbbd1a5ddb496ba64166d67da","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.582Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-6aa627df94a58271baa523c6: done: 0.018857 sec","job_status":"done","queue_duration_s":0.935381,"scheduling_latency_s":0.935421,"redis_calls":12,"redis_duration_s":0.00394,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000452,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002304,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000343,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000235785,"duration_s":0.018857,"completed_at":"2026-09-06T16:08:04.537Z","load_balancing_strategy":"primary","db_duration_s":0.00041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.284681] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0d7a47433f5f33cb8a8e4314","created_at":"2026-09-06T16:08:03.596Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f09c582d8f762822ab9dee37f7201c17","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.609Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0d7a47433f5f33cb8a8e4314: start","job_status":"start","queue_duration_s":0.929276,"scheduling_latency_s":0.929295} gitlab # [ 164.285485] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0d7a47433f5f33cb8a8e4314","created_at":"2026-09-06T16:08:03.596Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f09c582d8f762822ab9dee37f7201c17","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.609Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0d7a47433f5f33cb8a8e4314: done: 0.016949 sec","job_status":"done","queue_duration_s":0.929276,"scheduling_latency_s":0.929295,"redis_calls":10,"redis_duration_s":0.001245,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000204216,"duration_s":0.016949,"completed_at":"2026-09-06T16:08:04.555Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001693,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.330762] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.556Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a6461f2f7f8f85f014a074d6","created_at":"2026-09-06T16:08:03.622Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cf1e28a8beda196e7b277a0c9baae83","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.635Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a6461f2f7f8f85f014a074d6: start","job_status":"start","queue_duration_s":0.920888,"scheduling_latency_s":0.920902} gitlab # [ 164.331599] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.576Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a6461f2f7f8f85f014a074d6","created_at":"2026-09-06T16:08:03.622Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-66876ef4941e44db","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cf1e28a8beda196e7b277a0c9baae83","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:03.635Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a6461f2f7f8f85f014a074d6: done: 0.020226 sec","job_status":"done","queue_duration_s":0.920888,"scheduling_latency_s":0.920902,"redis_calls":12,"redis_duration_s":0.001582,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000624,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000312889,"exclusive_lock_hold_duration_s":0.008071696,"duration_s":0.020226,"completed_at":"2026-09-06T16:08:04.576Z","load_balancing_strategy":"primary","db_duration_s":0.005337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.345560] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c838bb103bfbedd1112bbd04","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c838bb103bfbedd1112bbd04: start","job_status":"start","queue_duration_s":0.514146,"scheduling_latency_s":0.51417} gitlab # [ 164.356248] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c838bb103bfbedd1112bbd04","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c838bb103bfbedd1112bbd04: done: 0.013633 sec","job_status":"done","queue_duration_s":0.514146,"scheduling_latency_s":0.51417,"redis_calls":16,"redis_duration_s":0.002025,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000189689,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013633,"completed_at":"2026-09-06T16:08:04.590Z","load_balancing_strategy":"primary","db_duration_s":0.000235,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.427852] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4c2e708bc6db55e3a4064bb","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4c2e708bc6db55e3a4064bb: start","job_status":"start","queue_duration_s":0.528628,"scheduling_latency_s":0.528664} gitlab # [ 164.428680] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4c2e708bc6db55e3a4064bb","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4c2e708bc6db55e3a4064bb: done: 0.009415 sec","job_status":"done","queue_duration_s":0.528628,"scheduling_latency_s":0.528664,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009415,"completed_at":"2026-09-06T16:08:04.601Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.435278] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d8440766b12708dfca8dae3","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d8440766b12708dfca8dae3: start","job_status":"start","queue_duration_s":0.538924,"scheduling_latency_s":0.538937} gitlab # [ 164.478753] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d8440766b12708dfca8dae3","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d8440766b12708dfca8dae3: done: 0.008975 sec","job_status":"done","queue_duration_s":0.538924,"scheduling_latency_s":0.538937,"redis_calls":13,"redis_duration_s":0.001514,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008975,"completed_at":"2026-09-06T16:08:04.611Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.479859] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7a44ebe05cf5f9180759488","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7a44ebe05cf5f9180759488: start","job_status":"start","queue_duration_s":0.55054,"scheduling_latency_s":0.550553} gitlab # [ 164.489588] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7a44ebe05cf5f9180759488","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7a44ebe05cf5f9180759488: done: 0.009127 sec","job_status":"done","queue_duration_s":0.55054,"scheduling_latency_s":0.550553,"redis_calls":13,"redis_duration_s":0.001631,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000464,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009127,"completed_at":"2026-09-06T16:08:04.622Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.528993] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59c0dfab2edfeda37f008480","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59c0dfab2edfeda37f008480: start","job_status":"start","queue_duration_s":0.560481,"scheduling_latency_s":0.560494} gitlab # [ 164.532770] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59c0dfab2edfeda37f008480","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59c0dfab2edfeda37f008480: done: 0.008887 sec","job_status":"done","queue_duration_s":0.560481,"scheduling_latency_s":0.560494,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008887,"completed_at":"2026-09-06T16:08:04.632Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.582027] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cec0ceb3826ef124fdb139b","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cec0ceb3826ef124fdb139b: start","job_status":"start","queue_duration_s":0.570188,"scheduling_latency_s":0.570201} gitlab # [ 164.582752] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cec0ceb3826ef124fdb139b","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cec0ceb3826ef124fdb139b: done: 0.008859 sec","job_status":"done","queue_duration_s":0.570188,"scheduling_latency_s":0.570201,"redis_calls":13,"redis_duration_s":0.0014749999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008859,"completed_at":"2026-09-06T16:08:04.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.620175] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70952d212dbe1cd219b445b5","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70952d212dbe1cd219b445b5: start","job_status":"start","queue_duration_s":0.579853,"scheduling_latency_s":0.579866} gitlab # [ 164.621068] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70952d212dbe1cd219b445b5","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70952d212dbe1cd219b445b5: done: 0.019676 sec","job_status":"done","queue_duration_s":0.579853,"scheduling_latency_s":0.579866,"redis_calls":13,"redis_duration_s":0.011719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019676,"completed_at":"2026-09-06T16:08:04.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.658487] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b143cfb88d65bc96acbf83da","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b143cfb88d65bc96acbf83da: start","job_status":"start","queue_duration_s":0.601356,"scheduling_latency_s":0.601374} gitlab # [ 164.663476] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b143cfb88d65bc96acbf83da","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b143cfb88d65bc96acbf83da: done: 0.014098 sec","job_status":"done","queue_duration_s":0.601356,"scheduling_latency_s":0.601374,"redis_calls":13,"redis_duration_s":0.00288,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014098,"completed_at":"2026-09-06T16:08:04.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.702569] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8e4e3fdc4735dc0e9f8aef6","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8e4e3fdc4735dc0e9f8aef6: start","job_status":"start","queue_duration_s":0.617422,"scheduling_latency_s":0.617449} gitlab # [ 164.707143] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8e4e3fdc4735dc0e9f8aef6","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8e4e3fdc4735dc0e9f8aef6: done: 0.015528 sec","job_status":"done","queue_duration_s":0.617422,"scheduling_latency_s":0.617449,"redis_calls":13,"redis_duration_s":0.0035889999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015528,"completed_at":"2026-09-06T16:08:04.696Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.762738] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a08ce2d1f45e2d6ac04f6509","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a08ce2d1f45e2d6ac04f6509: start","job_status":"start","queue_duration_s":0.633958,"scheduling_latency_s":0.633978} gitlab # [ 164.763606] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a08ce2d1f45e2d6ac04f6509","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a08ce2d1f45e2d6ac04f6509: done: 0.015662 sec","job_status":"done","queue_duration_s":0.633958,"scheduling_latency_s":0.633978,"redis_calls":13,"redis_duration_s":0.006557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001146,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005411,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015662,"completed_at":"2026-09-06T16:08:04.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.803969] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5409f4d905358e294f05d03c","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5409f4d905358e294f05d03c: start","job_status":"start","queue_duration_s":0.652976,"scheduling_latency_s":0.652993} gitlab # [ 164.804678] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5409f4d905358e294f05d03c","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5409f4d905358e294f05d03c: done: 0.010589 sec","job_status":"done","queue_duration_s":0.652976,"scheduling_latency_s":0.652993,"redis_calls":13,"redis_duration_s":0.001751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010589,"completed_at":"2026-09-06T16:08:04.726Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.856720] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"530f059fe5e2a4a6d91de6cb","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-530f059fe5e2a4a6d91de6cb: start","job_status":"start","queue_duration_s":0.664422,"scheduling_latency_s":0.664435} gitlab # [ 164.857635] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"530f059fe5e2a4a6d91de6cb","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-530f059fe5e2a4a6d91de6cb: done: 0.009082 sec","job_status":"done","queue_duration_s":0.664422,"scheduling_latency_s":0.664435,"redis_calls":13,"redis_duration_s":0.001458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009082,"completed_at":"2026-09-06T16:08:04.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.871173] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4efea189a7e0d1c8af2afc76","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4efea189a7e0d1c8af2afc76: start","job_status":"start","queue_duration_s":0.674386,"scheduling_latency_s":0.6744} gitlab # [ 164.917677] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4efea189a7e0d1c8af2afc76","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4efea189a7e0d1c8af2afc76: done: 0.008923 sec","job_status":"done","queue_duration_s":0.674386,"scheduling_latency_s":0.6744,"redis_calls":13,"redis_duration_s":0.001408,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000461,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008923,"completed_at":"2026-09-06T16:08:04.746Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.918673] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4146203eef12fb42c53af92","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4146203eef12fb42c53af92: start","job_status":"start","queue_duration_s":0.684105,"scheduling_latency_s":0.684118} gitlab # [ 164.956321] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4146203eef12fb42c53af92","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4146203eef12fb42c53af92: done: 0.008784 sec","job_status":"done","queue_duration_s":0.684105,"scheduling_latency_s":0.684118,"redis_calls":13,"redis_duration_s":0.001412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008784,"completed_at":"2026-09-06T16:08:04.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.958180] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"688d47396f9d39271e3015bb","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-688d47396f9d39271e3015bb: start","job_status":"start","queue_duration_s":0.696686,"scheduling_latency_s":0.6967} gitlab # [ 165.001932] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"688d47396f9d39271e3015bb","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-688d47396f9d39271e3015bb: done: 0.008982 sec","job_status":"done","queue_duration_s":0.696686,"scheduling_latency_s":0.6967,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008982,"completed_at":"2026-09-06T16:08:04.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.003539] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0074d39c911d3db7c6997dc3","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0074d39c911d3db7c6997dc3: start","job_status":"start","queue_duration_s":0.706508,"scheduling_latency_s":0.70652} gitlab # [ 165.013818] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0074d39c911d3db7c6997dc3","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0074d39c911d3db7c6997dc3: done: 0.008865 sec","job_status":"done","queue_duration_s":0.706508,"scheduling_latency_s":0.70652,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000455,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008865,"completed_at":"2026-09-06T16:08:04.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.056668] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"871657525c0e02d95d353aa8","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-871657525c0e02d95d353aa8: start","job_status":"start","queue_duration_s":0.716184,"scheduling_latency_s":0.716197} gitlab # [ 165.057601] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"871657525c0e02d95d353aa8","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-871657525c0e02d95d353aa8: done: 0.008818 sec","job_status":"done","queue_duration_s":0.716184,"scheduling_latency_s":0.716197,"redis_calls":13,"redis_duration_s":0.001466,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008818,"completed_at":"2026-09-06T16:08:04.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.114670] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3286357b3040815da9a07cc2","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3286357b3040815da9a07cc2: start","job_status":"start","queue_duration_s":0.725803,"scheduling_latency_s":0.725815} gitlab # [ 165.115796] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3286357b3040815da9a07cc2","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3286357b3040815da9a07cc2: done: 0.008711 sec","job_status":"done","queue_duration_s":0.725803,"scheduling_latency_s":0.725815,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008711,"completed_at":"2026-09-06T16:08:04.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.157325] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"362bb58bff97ddef31ce5ad9","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-362bb58bff97ddef31ce5ad9: start","job_status":"start","queue_duration_s":0.739937,"scheduling_latency_s":0.739951} gitlab # [ 165.158252] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"362bb58bff97ddef31ce5ad9","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-362bb58bff97ddef31ce5ad9: done: 0.00977 sec","job_status":"done","queue_duration_s":0.739937,"scheduling_latency_s":0.739951,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00977,"completed_at":"2026-09-06T16:08:04.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.201485] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7154cf1fe638c61316631f0e","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7154cf1fe638c61316631f0e: start","job_status":"start","queue_duration_s":0.750497,"scheduling_latency_s":0.75051} gitlab # [ 165.202403] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7154cf1fe638c61316631f0e","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7154cf1fe638c61316631f0e: done: 0.008789 sec","job_status":"done","queue_duration_s":0.750497,"scheduling_latency_s":0.75051,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008789,"completed_at":"2026-09-06T16:08:04.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.228915] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4933d69190b674495e32994","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4933d69190b674495e32994: start","job_status":"start","queue_duration_s":0.760072,"scheduling_latency_s":0.760085} gitlab # [ 165.240792] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4933d69190b674495e32994","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4933d69190b674495e32994: done: 0.00892 sec","job_status":"done","queue_duration_s":0.760072,"scheduling_latency_s":0.760085,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00892,"completed_at":"2026-09-06T16:08:04.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.287798] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1437cf52d5f4fa6c7f4de70","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1437cf52d5f4fa6c7f4de70: start","job_status":"start","queue_duration_s":0.769811,"scheduling_latency_s":0.769824} gitlab # [ 165.288826] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1437cf52d5f4fa6c7f4de70","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1437cf52d5f4fa6c7f4de70: done: 0.008985 sec","job_status":"done","queue_duration_s":0.769811,"scheduling_latency_s":0.769824,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008985,"completed_at":"2026-09-06T16:08:04.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.348922] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"479ceca7cec910bd75ad73aa","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-479ceca7cec910bd75ad73aa: start","job_status":"start","queue_duration_s":0.783802,"scheduling_latency_s":0.783817} gitlab # [ 165.349847] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"479ceca7cec910bd75ad73aa","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-479ceca7cec910bd75ad73aa: done: 0.010635 sec","job_status":"done","queue_duration_s":0.783802,"scheduling_latency_s":0.783817,"redis_calls":13,"redis_duration_s":0.001462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010635,"completed_at":"2026-09-06T16:08:04.857Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.398137] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91f9500eb0f3f2e8fdec0376","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91f9500eb0f3f2e8fdec0376: start","job_status":"start","queue_duration_s":0.795343,"scheduling_latency_s":0.795356} gitlab # [ 165.399161] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91f9500eb0f3f2e8fdec0376","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91f9500eb0f3f2e8fdec0376: done: 0.009278 sec","job_status":"done","queue_duration_s":0.795343,"scheduling_latency_s":0.795356,"redis_calls":13,"redis_duration_s":0.001449,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009278,"completed_at":"2026-09-06T16:08:04.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.417658] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a05d8081937dbf7459011c93","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a05d8081937dbf7459011c93: start","job_status":"start","queue_duration_s":0.805467,"scheduling_latency_s":0.805482} gitlab # [ 165.464940] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a05d8081937dbf7459011c93","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a05d8081937dbf7459011c93: done: 0.012083 sec","job_status":"done","queue_duration_s":0.805467,"scheduling_latency_s":0.805482,"redis_calls":13,"redis_duration_s":0.004264,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003611,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012083,"completed_at":"2026-09-06T16:08:04.880Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.466546] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28e7bc0471dc9d7e1d477364","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28e7bc0471dc9d7e1d477364: start","job_status":"start","queue_duration_s":0.818403,"scheduling_latency_s":0.818417} gitlab # [ 165.511817] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28e7bc0471dc9d7e1d477364","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28e7bc0471dc9d7e1d477364: done: 0.017006 sec","job_status":"done","queue_duration_s":0.818403,"scheduling_latency_s":0.818417,"redis_calls":13,"redis_duration_s":0.008977,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003259,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005718,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017006,"completed_at":"2026-09-06T16:08:04.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.513300] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b759becd3bcef44da568eee8","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b759becd3bcef44da568eee8: start","job_status":"start","queue_duration_s":0.836259,"scheduling_latency_s":0.836274} gitlab # [ 165.554680] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b759becd3bcef44da568eee8","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b759becd3bcef44da568eee8: done: 0.009193 sec","job_status":"done","queue_duration_s":0.836259,"scheduling_latency_s":0.836274,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009193,"completed_at":"2026-09-06T16:08:04.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.555585] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200987689a2564b110333746","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200987689a2564b110333746: start","job_status":"start","queue_duration_s":0.846276,"scheduling_latency_s":0.846289} gitlab # [ 165.565681] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200987689a2564b110333746","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200987689a2564b110333746: done: 0.008948 sec","job_status":"done","queue_duration_s":0.846276,"scheduling_latency_s":0.846289,"redis_calls":13,"redis_duration_s":0.0014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000461,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008948,"completed_at":"2026-09-06T16:08:04.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.603790] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5d5c2abe8558f75e895b6d4","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5d5c2abe8558f75e895b6d4: start","job_status":"start","queue_duration_s":0.856078,"scheduling_latency_s":0.856092} gitlab # [ 165.645913] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5d5c2abe8558f75e895b6d4","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5d5c2abe8558f75e895b6d4: done: 0.009141 sec","job_status":"done","queue_duration_s":0.856078,"scheduling_latency_s":0.856092,"redis_calls":13,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009141,"completed_at":"2026-09-06T16:08:04.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.647588] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c253cb2bc812eb569d627de","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c253cb2bc812eb569d627de: start","job_status":"start","queue_duration_s":0.866033,"scheduling_latency_s":0.866046} gitlab # [ 165.692224] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c253cb2bc812eb569d627de","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c253cb2bc812eb569d627de: done: 0.019165 sec","job_status":"done","queue_duration_s":0.866033,"scheduling_latency_s":0.866046,"redis_calls":13,"redis_duration_s":0.011246,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001787,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009459,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019165,"completed_at":"2026-09-06T16:08:04.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.693710] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4f873dbcddfc1c6fadac889","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4f873dbcddfc1c6fadac889: start","job_status":"start","queue_duration_s":0.887889,"scheduling_latency_s":0.887904} gitlab # [ 165.699564] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4f873dbcddfc1c6fadac889","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4f873dbcddfc1c6fadac889: done: 0.018451 sec","job_status":"done","queue_duration_s":0.887889,"scheduling_latency_s":0.887904,"redis_calls":13,"redis_duration_s":0.009651,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002356,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018451,"completed_at":"2026-09-06T16:08:04.969Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.741576] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6b1223dfb9cf000c4f0ce59","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6b1223dfb9cf000c4f0ce59: start","job_status":"start","queue_duration_s":0.910062,"scheduling_latency_s":0.910076} gitlab # [ 165.743557] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6b1223dfb9cf000c4f0ce59","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6b1223dfb9cf000c4f0ce59: done: 0.017615 sec","job_status":"done","queue_duration_s":0.910062,"scheduling_latency_s":0.910076,"redis_calls":13,"redis_duration_s":0.009061,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017615,"completed_at":"2026-09-06T16:08:04.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.782587] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:04.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60f621ed72ff90c90b10d7e7","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60f621ed72ff90c90b10d7e7: start","job_status":"start","queue_duration_s":0.928554,"scheduling_latency_s":0.928569} gitlab # [ 165.787764] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60f621ed72ff90c90b10d7e7","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60f621ed72ff90c90b10d7e7: done: 0.009051 sec","job_status":"done","queue_duration_s":0.928554,"scheduling_latency_s":0.928569,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009051,"completed_at":"2026-09-06T16:08:05.000Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.832635] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f84376511d85ac419aa5278e","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f84376511d85ac419aa5278e: start","job_status":"start","queue_duration_s":0.938455,"scheduling_latency_s":0.938469} gitlab # [ 165.833524] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f84376511d85ac419aa5278e","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f84376511d85ac419aa5278e: done: 0.009165 sec","job_status":"done","queue_duration_s":0.938455,"scheduling_latency_s":0.938469,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009165,"completed_at":"2026-09-06T16:08:05.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.877773] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3863feb6a39b7181862279fe","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3863feb6a39b7181862279fe: start","job_status":"start","queue_duration_s":0.948466,"scheduling_latency_s":0.948479} gitlab # [ 165.878507] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3863feb6a39b7181862279fe","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3863feb6a39b7181862279fe: done: 0.009262 sec","job_status":"done","queue_duration_s":0.948466,"scheduling_latency_s":0.948479,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009262,"completed_at":"2026-09-06T16:08:05.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.925772] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d2c494ca2b970282a324c0f","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d2c494ca2b970282a324c0f: start","job_status":"start","queue_duration_s":0.958576,"scheduling_latency_s":0.95859} gitlab # [ 165.926654] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d2c494ca2b970282a324c0f","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d2c494ca2b970282a324c0f: done: 0.014713 sec","job_status":"done","queue_duration_s":0.958576,"scheduling_latency_s":0.95859,"redis_calls":13,"redis_duration_s":0.006763,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005475,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014713,"completed_at":"2026-09-06T16:08:05.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.965456] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e77547d4e97a5fc716fa24d","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e77547d4e97a5fc716fa24d: start","job_status":"start","queue_duration_s":0.975908,"scheduling_latency_s":0.975923} gitlab # [ 165.966097] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e77547d4e97a5fc716fa24d","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e77547d4e97a5fc716fa24d: done: 0.010596 sec","job_status":"done","queue_duration_s":0.975908,"scheduling_latency_s":0.975923,"redis_calls":13,"redis_duration_s":0.0018930000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001358,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010596,"completed_at":"2026-09-06T16:08:05.049Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.019972] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"592012e451989c7ecb7d549d","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-592012e451989c7ecb7d549d: start","job_status":"start","queue_duration_s":0.987359,"scheduling_latency_s":0.987373} gitlab # [ 166.020671] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"592012e451989c7ecb7d549d","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-592012e451989c7ecb7d549d: done: 0.009022 sec","job_status":"done","queue_duration_s":0.987359,"scheduling_latency_s":0.987373,"redis_calls":13,"redis_duration_s":0.001408,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000464,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009022,"completed_at":"2026-09-06T16:08:05.059Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.063251] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1384f91da81c64563c94b3c2","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1384f91da81c64563c94b3c2: start","job_status":"start","queue_duration_s":0.997197,"scheduling_latency_s":0.99721} gitlab # [ 166.063957] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1384f91da81c64563c94b3c2","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1384f91da81c64563c94b3c2: done: 0.008701 sec","job_status":"done","queue_duration_s":0.997197,"scheduling_latency_s":0.99721,"redis_calls":13,"redis_duration_s":0.001436,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008701,"completed_at":"2026-09-06T16:08:05.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.087259] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbbcfae23a694f202d715c1e","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbbcfae23a694f202d715c1e: start","job_status":"start","queue_duration_s":1.006765,"scheduling_latency_s":1.006778} gitlab # [ 166.098139] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbbcfae23a694f202d715c1e","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbbcfae23a694f202d715c1e: done: 0.008804 sec","job_status":"done","queue_duration_s":1.006765,"scheduling_latency_s":1.006778,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008804,"completed_at":"2026-09-06T16:08:05.078Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.136632] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2cfafa38736bbac10ca13b6","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2cfafa38736bbac10ca13b6: start","job_status":"start","queue_duration_s":1.021922,"scheduling_latency_s":1.021936} gitlab # [ 166.143468] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2cfafa38736bbac10ca13b6","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2cfafa38736bbac10ca13b6: done: 0.012134 sec","job_status":"done","queue_duration_s":1.021922,"scheduling_latency_s":1.021936,"redis_calls":13,"redis_duration_s":0.003801,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012134,"completed_at":"2026-09-06T16:08:05.097Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.181058] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"301433f6089ef3e8fbbcc205","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-301433f6089ef3e8fbbcc205: start","job_status":"start","queue_duration_s":1.034867,"scheduling_latency_s":1.034881} gitlab # [ 166.187971] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"301433f6089ef3e8fbbcc205","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-301433f6089ef3e8fbbcc205: done: 0.008594 sec","job_status":"done","queue_duration_s":1.034867,"scheduling_latency_s":1.034881,"redis_calls":13,"redis_duration_s":0.001381,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008594,"completed_at":"2026-09-06T16:08:05.106Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.230462] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8890c752d5b4ec2d5b7be90b","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8890c752d5b4ec2d5b7be90b: start","job_status":"start","queue_duration_s":1.044263,"scheduling_latency_s":1.044277} gitlab # [ 166.232957] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8890c752d5b4ec2d5b7be90b","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8890c752d5b4ec2d5b7be90b: done: 0.008513 sec","job_status":"done","queue_duration_s":1.044263,"scheduling_latency_s":1.044277,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000446,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008513,"completed_at":"2026-09-06T16:08:05.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.276050] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57fe0d400e22b3c2dcb431e9","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57fe0d400e22b3c2dcb431e9: start","job_status":"start","queue_duration_s":1.053542,"scheduling_latency_s":1.053555} gitlab # [ 166.278283] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57fe0d400e22b3c2dcb431e9","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57fe0d400e22b3c2dcb431e9: done: 0.008596 sec","job_status":"done","queue_duration_s":1.053542,"scheduling_latency_s":1.053555,"redis_calls":13,"redis_duration_s":0.001365,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000454,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008596,"completed_at":"2026-09-06T16:08:05.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.329994] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75c33fb99832eab51c9d846a","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75c33fb99832eab51c9d846a: start","job_status":"start","queue_duration_s":1.062954,"scheduling_latency_s":1.062966} gitlab # [ 166.330907] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75c33fb99832eab51c9d846a","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75c33fb99832eab51c9d846a: done: 0.019175 sec","job_status":"done","queue_duration_s":1.062954,"scheduling_latency_s":1.062966,"redis_calls":13,"redis_duration_s":0.011662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019175,"completed_at":"2026-09-06T16:08:05.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.371127] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7afa368f62d316b180301f77","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7afa368f62d316b180301f77: start","job_status":"start","queue_duration_s":1.083229,"scheduling_latency_s":1.083243} gitlab # [ 166.371984] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7afa368f62d316b180301f77","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7afa368f62d316b180301f77: done: 0.008684 sec","job_status":"done","queue_duration_s":1.083229,"scheduling_latency_s":1.083243,"redis_calls":13,"redis_duration_s":0.0014689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008684,"completed_at":"2026-09-06T16:08:05.155Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.416519] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"085e107d47489117ca715683","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-085e107d47489117ca715683: start","job_status":"start","queue_duration_s":1.092754,"scheduling_latency_s":1.092767} gitlab # [ 166.417403] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"085e107d47489117ca715683","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-085e107d47489117ca715683: done: 0.008568 sec","job_status":"done","queue_duration_s":1.092754,"scheduling_latency_s":1.092767,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008568,"completed_at":"2026-09-06T16:08:05.164Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.467666] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b32e7fa5a85d0684e05b4376","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b32e7fa5a85d0684e05b4376: start","job_status":"start","queue_duration_s":1.102203,"scheduling_latency_s":1.102216} gitlab # [ 166.468611] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b32e7fa5a85d0684e05b4376","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b32e7fa5a85d0684e05b4376: done: 0.008559 sec","job_status":"done","queue_duration_s":1.102203,"scheduling_latency_s":1.102216,"redis_calls":13,"redis_duration_s":0.0014470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008559,"completed_at":"2026-09-06T16:08:05.173Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.507918] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91aa888d232356530d5ce6d2","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91aa888d232356530d5ce6d2: start","job_status":"start","queue_duration_s":1.111567,"scheduling_latency_s":1.111579} gitlab # [ 166.508660] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91aa888d232356530d5ce6d2","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91aa888d232356530d5ce6d2: done: 0.011401 sec","job_status":"done","queue_duration_s":1.111567,"scheduling_latency_s":1.111579,"redis_calls":13,"redis_duration_s":0.003819,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003335,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011401,"completed_at":"2026-09-06T16:08:05.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.548735] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d38fbd5639281502fd576712","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d38fbd5639281502fd576712: start","job_status":"start","queue_duration_s":1.123809,"scheduling_latency_s":1.123822} gitlab # [ 166.550069] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d38fbd5639281502fd576712","created_at":"2026-09-06T16:08:04.017Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-fe6d231ccc05422c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4ae534acfe3158ea46c185a9c5a31c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.063Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d38fbd5639281502fd576712: done: 0.045619 sec","job_status":"done","queue_duration_s":1.123809,"scheduling_latency_s":1.123822,"redis_calls":13,"redis_duration_s":0.001913,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.045619,"completed_at":"2026-09-06T16:08:05.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.616230] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.233Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5f4d944fb0a7a46f47dd0ead","created_at":"2026-09-06T16:08:04.500Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-bb7ee187488e460c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.511Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5f4d944fb0a7a46f47dd0ead: start","job_status":"start","queue_duration_s":0.72267,"scheduling_latency_s":0.72269} gitlab # [ 166.616950] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.246Z","message":"No existing merge request to be cleaned up."} gitlab # [ 166.617820] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.255Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5f4d944fb0a7a46f47dd0ead","created_at":"2026-09-06T16:08:04.500Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-bb7ee187488e460c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.511Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5f4d944fb0a7a46f47dd0ead: done: 0.02151 sec","job_status":"done","queue_duration_s":0.72267,"scheduling_latency_s":0.72269,"redis_calls":13,"redis_duration_s":0.002762,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000413,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001441,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000234946,"duration_s":0.02151,"completed_at":"2026-09-06T16:08:05.255Z","load_balancing_strategy":"primary","db_duration_s":0.004301,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.662835] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.256Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bd9938334ec2dc6f02290e43","created_at":"2026-09-06T16:08:04.500Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-bb7ee187488e460c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.511Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bd9938334ec2dc6f02290e43: start","job_status":"start","queue_duration_s":0.745018,"scheduling_latency_s":0.745031} gitlab # [ 166.663655] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.267Z","message":"No existing merge request to be cleaned up."} gitlab # [ 166.664137] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.275Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bd9938334ec2dc6f02290e43","created_at":"2026-09-06T16:08:04.500Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-bb7ee187488e460c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.511Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bd9938334ec2dc6f02290e43: done: 0.019016 sec","job_status":"done","queue_duration_s":0.745018,"scheduling_latency_s":0.745031,"redis_calls":10,"redis_duration_s":0.008359,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004325,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019016,"completed_at":"2026-09-06T16:08:05.275Z","load_balancing_strategy":"primary","db_duration_s":0.001107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.701938] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.276Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"96215939e21fa4e3c8c3229c","created_at":"2026-09-06T16:08:04.500Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-bb7ee187488e460c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.511Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-96215939e21fa4e3c8c3229c: start","job_status":"start","queue_duration_s":0.764903,"scheduling_latency_s":0.764917} gitlab # [ 166.702719] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.286Z","message":"No existing merge request to be cleaned up."} gitlab # [ 166.703574] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.294Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"96215939e21fa4e3c8c3229c","created_at":"2026-09-06T16:08:04.500Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-bb7ee187488e460c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.511Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-96215939e21fa4e3c8c3229c: done: 0.018345 sec","job_status":"done","queue_duration_s":0.764903,"scheduling_latency_s":0.764917,"redis_calls":10,"redis_duration_s":0.001927,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001348,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018345,"completed_at":"2026-09-06T16:08:05.294Z","load_balancing_strategy":"primary","db_duration_s":0.005529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.749646] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.296Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"628319b6a5f98fe727b01e31","created_at":"2026-09-06T16:08:04.500Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-bb7ee187488e460c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.511Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-628319b6a5f98fe727b01e31: start","job_status":"start","queue_duration_s":0.78494,"scheduling_latency_s":0.78496} gitlab # [ 166.750464] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.306Z","message":"No existing merge request to be cleaned up."} gitlab # [ 166.750919] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:05.315Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"628319b6a5f98fe727b01e31","created_at":"2026-09-06T16:08:04.500Z","trace_propagation_headers":{"sentry-trace":"69809a48c19f423baff0bb95c2e6fbf9-bb7ee187488e460c","baggage":"sentry-trace_id=69809a48c19f423baff0bb95c2e6fbf9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9ccf08bc60f0d8bf44227fdbdad8d003","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:04.511Z","job_size_bytes":2,"pid":1706,"sidekiq_tid":"maiq","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-628319b6a5f98fe727b01e31: done: 0.01912 sec","job_status":"done","queue_duration_s":0.78494,"scheduling_latency_s":0.78496,"redis_calls":10,"redis_duration_s":0.005047,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003885,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01912,"completed_at":"2026-09-06T16:08:05.315Z","load_balancing_strategy":"primary","db_duration_s":0.004422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.955172] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2543007098 wd_nsec: 2543006804 gitlab # [ 177.980867] setup-gitlab-start[2352]: W, [2026-09-06T16:08:18.580228 #2352] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 204.454188] postgres[2582]: [2582] ERROR: relation "table_without_models" does not exist gitlab # [ 204.455778] postgres[2582]: [2582] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 207.758237] runuser[2351]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 207.761876] setup-gitlab-start[2328]: + systemctl restart gitlab-sidekiq.service gitlab # [ 207.781236] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 207.781697] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:48.381Z","message":"Shutting down"} gitlab # [ 207.782993] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:48.383Z","message":"Scheduler exiting..."} gitlab # [ 207.788888] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:48.384Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 207.789360] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:48.384Z","message":"Scheduler exiting..."} gitlab # [ 208.290228] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:48.890Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 210.293942] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:50.893Z","message":"Bye!"} gitlab # [ 210.324154] sidekiq-cluster[1706]: {"severity":"INFO","time":"2026-09-06T16:08:50.924Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1706,"worker_id":"sidekiq_0","memwd_rss_bytes":940093440,"retry":0} gitlab # [ 210.923334] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 210.926176] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 210.928186] systemd[1]: gitlab-sidekiq.service: Consumed 1min 5.095s CPU time over 2min 9.378s wall clock time, 959.1M memory peak, 4.4M read from disk. gitlab # [ 210.932914] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 210.935483] setup-gitlab-start[2328]: + mkdir -p /var/lib/secrets gitlab # [ 210.937633] setup-gitlab-start[2328]: + echo nixbot-test-token-of-20-chars gitlab # [ 210.939847] setup-gitlab-start[2328]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 210.942281] setup-gitlab-start[2328]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 211.894441] gitaly[893]: time="2026-09-06T16:08:52.494Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.494" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.907083] gitaly[893]: time="2026-09-06T16:08:52.499Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.494" grpc.time_ms=4.326 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.035687] gitaly[893]: time="2026-09-06T16:08:52.636Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.145" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.636" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.045379] gitaly[893]: time="2026-09-06T16:08:52.636Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.145" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.636" grpc.time_ms=0.374 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.105568] gitaly[893]: time="2026-09-06T16:08:52.706Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M diskcache=f7250edf-9266-4b1c-ad43-a23d17d7e678 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:09:02.166" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.656" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.115429] gitaly[893]: time="2026-09-06T16:08:52.706Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1580 command.oublock=32 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:09:02.166" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.656" grpc.time_ms=49.816 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.128103] gitaly[893]: time="2026-09-06T16:08:52.723Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.232" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.723" pid=893 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 212.137550] gitaly[893]: time="2026-09-06T16:08:52.724Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.232" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.723" grpc.time_ms=0.703 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.147477] gitaly[893]: time="2026-09-06T16:08:52.748Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:09:02.238" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.728" grpc.time_ms=9.91 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.200542] gitaly[893]: time="2026-09-06T16:08:52.801Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.800" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.210705] gitaly[893]: time="2026-09-06T16:08:52.801Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.800" grpc.time_ms=0.788 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.246507] gitaly[893]: time="2026-09-06T16:08:52.847Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M diskcache=f6ee0580-a809-4b5c-b2b1-3e07effe80b9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:09:02.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.803" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.256163] gitaly[893]: time="2026-09-06T16:08:52.856Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1578 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:09:02.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.803" grpc.time_ms=44.101 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.275466] gitaly[893]: time="2026-09-06T16:08:52.876Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.085" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.875" pid=893 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 212.286094] gitaly[893]: time="2026-09-06T16:08:52.876Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.085" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.875" grpc.time_ms=0.344 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.298642] gitaly[893]: time="2026-09-06T16:08:52.899Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:09:02.096" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.886" grpc.time_ms=13.069 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.323580] gitaly[893]: time="2026-09-06T16:08:52.924Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:09:02.133" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.923" grpc.time_ms=0.43 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.334598] gitaly[893]: time="2026-09-06T16:08:52.935Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:09:02.144" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:52.935" grpc.time_ms=0.202 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.756958] gitaly[893]: time="2026-09-06T16:08:53.357Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNK364MK1ABHS1VY6CY6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:09:03.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:53.348" grpc.time_ms=9.322 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.185158] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:53 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 2227 gitlab # [ 213.192256] setup-gitlab-start[2328]: + mkdir -p /tmp/test-flake gitlab # [ 213.196760] setup-gitlab-start[2328]: + cd /tmp/test-flake gitlab # [ 213.197570] setup-gitlab-start[2328]: + git init -b master gitlab # [ 213.224856] setup-gitlab-start[2673]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 213.225714] setup-gitlab-start[2328]: + git config user.name test gitlab # [ 213.233413] setup-gitlab-start[2328]: + git config user.email test@example.com gitlab # [ 213.239734] setup-gitlab-start[2328]: + cat gitlab # [ 213.244251] setup-gitlab-start[2328]: + git add flake.nix gitlab # [ 213.249781] setup-gitlab-start[2328]: + git commit -m 'initial commit' gitlab # [ 213.274947] setup-gitlab-start[2678]: [master (root-commit) f12cff8] initial commit gitlab # [ 213.275581] setup-gitlab-start[2678]: 1 file changed, 10 insertions(+) gitlab # [ 213.277662] setup-gitlab-start[2678]: create mode 100644 flake.nix gitlab # [ 213.278533] setup-gitlab-start[2328]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 213.289931] setup-gitlab-start[2328]: + git push -u origin master gitlab # [ 213.453653] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:54 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 132 gitlab # [ 213.657200] gitaly[893]: time="2026-09-06T16:08:54.257Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNH8C8PEQE3184VX9B5V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:54.250" grpc.time_ms=6.614 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.657801] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:54 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 201 gitlab # [ 214.175865] gitaly[893]: time="2026-09-06T16:08:54.776Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=11 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:09:04.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:54.758" grpc.time_ms=17.845 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.249499] gitaly[893]: time="2026-09-06T16:08:54.850Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=8 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:09:24.034" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:08:54.834" grpc.time_ms=15.324 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.327860] gitaly[893]: time="2026-09-06T16:08:54.928Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW duration_ms=367ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 214.373260] gitaly[893]: time="2026-09-06T16:08:54.972Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW duration_ms=42ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 214.374980] gitaly[893]: time="2026-09-06T16:08:54.973Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5311 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:54.559" grpc.time_ms=414.064 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.469793] gitaly[893]: time="2026-09-06T16:08:55.070Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:55.059" grpc.time_ms=8.17 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.531268] gitaly[893]: time="2026-09-06T16:08:55.131Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:55.131" grpc.time_ms=0.712 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.596414] gitaly[893]: time="2026-09-06T16:08:55.193Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:55.192" grpc.time_ms=0.46 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.789765] gitaly[893]: time="2026-09-06T16:08:55.389Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:09:05.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:08:55.381" grpc.time_ms=7.902 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.825825] gitaly[893]: time="2026-09-06T16:08:55.426Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1569 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:09:05.025" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:08:55.405" grpc.time_ms=20.672 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.850939] gitaly[893]: time="2026-09-06T16:08:55.451Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW duration_ms=192ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 214.854579] gitaly[893]: time="2026-09-06T16:08:55.455Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4805 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:55.258" grpc.time_ms=196.636 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.872384] gitaly[893]: time="2026-09-06T16:08:55.472Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW diskcache=97090069-9a00-4e94-bc44-9370fcd2df93 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:54.403" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.873036] gitaly[893]: time="2026-09-06T16:08:55.473Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=251 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13467 command.oublock=32 command.real_time_ms=1063 command.spawn_token_fork_ms=4 command.system_time_ms=204 command.user_time_ms=47 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:54.403" grpc.time_ms=1069.963 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.874173] gitlab-workhorse[827]: time="2026-09-06T16:08:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 214.874812] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:55 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1177 gitlab # [ 214.899306] setup-gitlab-start[2682]: To http://gitlab/root/test-flake.git gitlab # [ 214.899505] setup-gitlab-start[2682]: * [new branch] master -> master gitlab # [ 214.904023] setup-gitlab-start[2682]: branch 'master' set up to track 'origin/master'. gitlab # [ 214.906736] systemd[1]: Finished setup-gitlab.service. gitlab # [ 214.909717] systemd[1]: Starting nixbot CI service... gitlab # [ 215.048212] sidekiq-cluster[2621]: {"severity":"INFO","time":"2026-09-06T16:08:55.587Z","message":"Starting cluster with 1 processes"} gitlab: (finished: waiting for unit setup-gitlab.service, in 215.86 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 216.290214] nixbot[2766]: {"ts": "2026-09-06T16:08:56.889Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 216.352204] nixbot[2766]: {"ts": "2026-09-06T16:08:56.952Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 216.385798] nixbot[2766]: {"ts": "2026-09-06T16:08:56.983Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 216.391244] nixbot[2766]: {"ts": "2026-09-06T16:08:56.991Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 216.399470] nixbot[2766]: {"ts": "2026-09-06T16:08:56.998Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 216.408545] nixbot[2766]: {"ts": "2026-09-06T16:08:57.009Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 216.436323] nixbot[2766]: {"ts": "2026-09-06T16:08:57.032Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 216.455158] nixbot[2766]: {"ts": "2026-09-06T16:08:57.054Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 216.471907] nixbot[2766]: {"ts": "2026-09-06T16:08:57.072Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 216.481078] nixbot[2766]: {"ts": "2026-09-06T16:08:57.081Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 216.489768] nixbot[2766]: {"ts": "2026-09-06T16:08:57.090Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 216.504451] nixbot[2766]: {"ts": "2026-09-06T16:08:57.102Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 216.505564] nixbot[2766]: {"ts": "2026-09-06T16:08:57.106Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 216.509245] nixbot[2766]: {"ts": "2026-09-06T16:08:57.108Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 216.511763] nixbot[2766]: {"ts": "2026-09-06T16:08:57.111Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 216.514572] nixbot[2766]: {"ts": "2026-09-06T16:08:57.114Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 216.527052] nixbot[2766]: {"ts": "2026-09-06T16:08:57.126Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 216.529476] nixbot[2766]: {"ts": "2026-09-06T16:08:57.129Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 216.534724] nixbot[2766]: {"ts": "2026-09-06T16:08:57.135Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 216.537088] nixbot[2766]: {"ts": "2026-09-06T16:08:57.137Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 216.542758] nixbot[2766]: {"ts": "2026-09-06T16:08:57.141Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 216.549452] nixbot[2766]: {"ts": "2026-09-06T16:08:57.149Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 216.553095] nixbot[2766]: {"ts": "2026-09-06T16:08:57.153Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 216.562235] nixbot[2766]: {"ts": "2026-09-06T16:08:57.162Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 216.566928] nixbot[2766]: {"ts": "2026-09-06T16:08:57.167Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 216.572974] nixbot[2766]: {"ts": "2026-09-06T16:08:57.172Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 216.580909] nixbot[2766]: {"ts": "2026-09-06T16:08:57.181Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 216.589316] nixbot[2766]: {"ts": "2026-09-06T16:08:57.189Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 216.595939] nixbot[2766]: {"ts": "2026-09-06T16:08:57.196Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 216.610687] nixbot[2766]: {"ts": "2026-09-06T16:08:57.209Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 216.617661] nixbot[2766]: {"ts": "2026-09-06T16:08:57.218Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 216.620375] nixbot[2766]: {"ts": "2026-09-06T16:08:57.221Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 216.624797] nixbot[2766]: {"ts": "2026-09-06T16:08:57.225Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "public_badges"} gitlab # [ 216.995761] nixbot[2766]: INFO: Started server process [2766] gitlab # [ 216.996830] nixbot[2766]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 217.020274] nixbot[2766]: INFO: ::ffff:127.0.0.1:40670 - "GET /health HTTP/1.1" 200 OK gitlab # [ 217.026232] systemd[1]: Started nixbot CI service. gitlab # [ 217.027453] systemd[1]: Reached target Multi-User System. gitlab # [ 217.028340] systemd[1]: Startup finished in 1.123s (kernel) + 3.367s (initrd) + 3min 32.537s (userspace) = 3min 37.028s. gitlab: (finished: waiting for unit nixbot.service, in 2.10 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 217.401459] nixbot[2766]: INFO: ::ffff:127.0.0.1:40686 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 217.474692] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 53 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds) gitlab # [ 217.755194] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:58 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 735 gitlab # [ 217.757559] nixbot[2766]: {"ts": "2026-09-06T16:08:58.357Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 217.786640] nixbot[2766]: {"ts": "2026-09-06T16:08:58.387Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 217.870563] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 65 gitlab # [ 217.875385] nixbot[2766]: {"ts": "2026-09-06T16:08:58.472Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 217.881634] nixbot[2766]: {"ts": "2026-09-06T16:08:58.473Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 217.986339] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 111 gitlab # [ 217.987234] nixbot[2766]: {"ts": "2026-09-06T16:08:58.587Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 218.112449] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:58 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 119 gitlab # [ 218.113459] nixbot[2766]: {"ts": "2026-09-06T16:08:58.714Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 218.169692] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 46 gitlab # [ 218.170817] nixbot[2766]: {"ts": "2026-09-06T16:08:58.771Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 218.225080] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:58 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 51 gitlab # [ 218.225916] nixbot[2766]: {"ts": "2026-09-06T16:08:58.826Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 218.300454] gitaly[893]: time="2026-09-06T16:08:58.899Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNT6F64RE5ZNQZ8S555BC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-06T16:09:28.086" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:08:58.887" grpc.time_ms=12.161 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.326057] gitaly[893]: time="2026-09-06T16:08:58.926Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNT6F64RE5ZNQZ8S555BC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:09:28.125" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:08:58.926" grpc.time_ms=0.539 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.338494] gitaly[893]: time="2026-09-06T16:08:58.938Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNT6F64RE5ZNQZ8S555BC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-06T16:09:08.179" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:08:58.930" grpc.time_ms=7.525 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.361751] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 130 gitlab # [ 218.362733] nixbot[2766]: {"ts": "2026-09-06T16:08:58.963Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 218.434193] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 68 gitlab # [ 218.435542] nixbot[2766]: {"ts": "2026-09-06T16:08:59.035Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 218.438309] nixbot[2766]: {"ts": "2026-09-06T16:08:59.038Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 218.523083] gitaly[893]: time="2026-09-06T16:08:59.123Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTDXRPPPNST7WG67YMHR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.115" grpc.time_ms=7.97 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.539279] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 68 gitlab # [ 218.582994] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 81 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds) (finished: subtest: project discovered and webhook registered, in 1.18 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 218.633536] gitaly[893]: time="2026-09-06T16:08:59.234Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTG6H49GSGE2J997TFPV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.226" pid=893 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 218.635020] gitaly[893]: time="2026-09-06T16:08:59.234Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTG6H49GSGE2J997TFPV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.226" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.636684] gitlab-workhorse[827]: time="2026-09-06T16:08:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 218.637523] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 94 gitlab # [ 218.645630] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 31 gitlab # [ 218.662925] gitaly[893]: time="2026-09-06T16:08:59.263Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTG6H49GSGE2J997TFPV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.226" grpc.time_ms=7.852 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.748199] gitaly[893]: time="2026-09-06T16:08:59.348Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTKY7SFZC271A6773X6F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.342" grpc.time_ms=6.115 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.765098] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 102 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done. gitlab # [ 218.779371] gitaly[893]: time="2026-09-06T16:08:59.380Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTM1V22YSD5RZ65JAK1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.367" pid=893 remote_ip="2001:db8:1::1" request_sha=ec620836e6fdf71f6eb26d0aa31f0ad5bfba9d93 response_bytes=228 span.kind=server system=grpc gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 218.779804] gitaly[893]: time="2026-09-06T16:08:59.380Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1206 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTM1V22YSD5RZ65JAK1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.367" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.781264] gitaly[893]: time="2026-09-06T16:08:59.380Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1206 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTM1V22YSD5RZ65JAK1Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.367" grpc.time_ms=12.539 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.803657] gitlab-workhorse[827]: time="2026-09-06T16:08:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=228 gitlab # [ 218.820111] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 140 gitlab # [ 218.915843] gitaly[893]: time="2026-09-06T16:08:59.516Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTT2389W8DH124SCVNHW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.508" grpc.time_ms=7.459 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.930607] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 72 gitlab # [ 218.980707] gitaly[893]: time="2026-09-06T16:08:59.581Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTWEJEP3BJ7EAY4CP40M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.573" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 218.981302] gitaly[893]: time="2026-09-06T16:08:59.581Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTWEJEP3BJ7EAY4CP40M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.573" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.982119] gitaly[893]: time="2026-09-06T16:08:59.581Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTWEJEP3BJ7EAY4CP40M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.573" grpc.time_ms=8.337 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.020255] gitlab-workhorse[827]: time="2026-09-06T16:08:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 219.020450] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 86 gitlab # [ 219.025737] gitaly[893]: time="2026-09-06T16:08:59.626Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=8 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:09:29.012" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:08:59.612" grpc.time_ms=13.537 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.049621] gitaly[893]: time="2026-09-06T16:08:59.650Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 duration_ms=76ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 219.069120] gitaly[893]: time="2026-09-06T16:08:59.669Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 duration_ms=19ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 219.069288] gitaly[893]: time="2026-09-06T16:08:59.670Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5311 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:59.573" grpc.time_ms=96.1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.106079] gitaly[893]: time="2026-09-06T16:08:59.706Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:59.705" grpc.time_ms=0.757 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.132430] gitaly[893]: time="2026-09-06T16:08:59.733Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNV0H6S7WM4W8YG4N597W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.726" grpc.time_ms=6.776 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.133180] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 67 gitlab # [ 219.155581] gitaly[893]: time="2026-09-06T16:08:59.754Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:59.754" grpc.time_ms=0.301 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.180139] gitaly[893]: time="2026-09-06T16:08:59.780Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:59.779" grpc.time_ms=0.526 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.225352] gitaly[893]: time="2026-09-06T16:08:59.826Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNV34ZB6YD93J6HQ996KA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.814" pid=893 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 219.225884] gitaly[893]: time="2026-09-06T16:08:59.826Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1177 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNV34ZB6YD93J6HQ996KA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.814" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.226956] gitaly[893]: time="2026-09-06T16:08:59.827Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1177 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNV34ZB6YD93J6HQ996KA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.814" grpc.time_ms=12.593 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.228918] gitlab-workhorse[827]: time="2026-09-06T16:08:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 219.231071] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 80 gitlab # [ 219.245030] gitaly[893]: time="2026-09-06T16:08:59.845Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 duration_ms=33ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 219.246222] gitaly[893]: time="2026-09-06T16:08:59.847Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4805 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:08:59.811" grpc.time_ms=35.198 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.328812] gitaly[893]: time="2026-09-06T16:08:59.929Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 diskcache=9d3eebc8-cb0d-4025-9b8d-d490d6314bc7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.530" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.329082] gitaly[893]: time="2026-09-06T16:08:59.929Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=139 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13336 command.oublock=32 command.real_time_ms=347 command.spawn_token_fork_ms=1 command.system_time_ms=92 command.user_time_ms=47 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:08:59.530" grpc.time_ms=399.242 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.350745] gitlab-workhorse[827]: time="2026-09-06T16:08:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 219.351299] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:08:59 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 528 gitlab # To http://gitlab/root/test-flake.git gitlab # f12cff8..f684670 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.78 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 219.429298] nixbot[2766]: INFO: ::ffff:127.0.0.1:40694 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab # [ 219.431481] gitaly[893]: time="2026-09-06T16:09:00.029Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:09:30.027" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:00.028" grpc.time_ms=1.33 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.08 seconds) gitlab # [ 219.456040] gitaly[893]: time="2026-09-06T16:09:00.056Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:09:30.049" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:00.050" grpc.time_ms=6.632 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.476591] gitaly[893]: time="2026-09-06T16:09:00.077Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:09:10.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:00.070" grpc.time_ms=6.792 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.491715] gitaly[893]: time="2026-09-06T16:09:00.091Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:09:10.024" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:00.085" grpc.time_ms=5.839 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.545447] gitaly[893]: time="2026-09-06T16:09:00.146Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:09:10.085" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:00.145" grpc.time_ms=0.199 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.562212] gitaly[893]: time="2026-09-06T16:09:00.162Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:09:10.096" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:00.157" grpc.time_ms=5.481 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.613301] gitaly[893]: time="2026-09-06T16:09:00.214Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4149 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:00.213" grpc.time_ms=0.388 pid=893 span.kind=server system=grpc gitlab # [ 219.638433] gitaly[893]: time="2026-09-06T16:09:00.239Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4151 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:00.238" grpc.time_ms=0.609 pid=893 span.kind=server system=grpc gitlab # [ 219.652662] gitaly[893]: time="2026-09-06T16:09:00.253Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y diskcache=5a98fe85-783d-4844-8cb6-44f17700cfb4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:09:10.118" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:00.178" pid=893 ref=refs/keep-around/f12cff88c90489b6aedd8947843349145d58ca39 remote_ip="2001:db8:1::1" revision=f12cff88c90489b6aedd8947843349145d58ca39 span.kind=server system=grpc username=root gitlab # [ 219.662655] gitaly[893]: time="2026-09-06T16:09:00.253Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=52 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5347 command.oublock=8 command.real_time_ms=68 command.spawn_token_fork_ms=0 command.system_time_ms=31 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:09:10.118" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:00.178" grpc.time_ms=74.905 pid=893 ref=refs/keep-around/f12cff88c90489b6aedd8947843349145d58ca39 remote_ip="2001:db8:1::1" revision=f12cff88c90489b6aedd8947843349145d58ca39 span.kind=server system=grpc username=root gitlab # [ 219.662962] gitaly[893]: time="2026-09-06T16:09:00.256Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:09:30.256" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:00.256" grpc.time_ms=0.286 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.777569] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 403 gitlab # [ 219.784412] nixbot[2766]: {"ts": "2026-09-06T16:09:00.384Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 \"HTTP/1.1 201 Created\""} gitlab # [ 219.799060] nixbot[2766]: {"ts": "2026-09-06T16:09:00.399Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-f12cff88c904-0"} gitlab # [ 219.954146] nixbot[2766]: {"ts": "2026-09-06T16:09:00.554Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-f12cff88c904-0", "argv0": "bwrap"} gitlab # [ 220.046896] systemd[1]: Started Nix Daemon. gitlab # [ 220.086881] nix-daemon[2996]: accepted connection from pid 2991, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 220.479116] nixbot[2766]: INFO: ::ffff:127.0.0.1:40698 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 221.514220] nixbot[2766]: INFO: ::ffff:127.0.0.1:40710 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 222.548084] nixbot[2766]: INFO: ::ffff:127.0.0.1:40726 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 223.587237] nixbot[2766]: INFO: ::ffff:127.0.0.1:50838 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 224.625344] nixbot[2766]: INFO: ::ffff:127.0.0.1:50844 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 224.991678] nix-daemon[2996]: accepted connection from pid 3022, user nixbot gitlab # [ 225.004976] nix-daemon[2996]: accepted connection from pid 2991, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 225.664460] nixbot[2766]: INFO: ::ffff:127.0.0.1:50846 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 226.705673] nixbot[2766]: INFO: ::ffff:127.0.0.1:50858 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 227.745234] nixbot[2766]: INFO: ::ffff:127.0.0.1:50864 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 228.783207] nixbot[2766]: INFO: ::ffff:127.0.0.1:50878 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 229.537543] nix-daemon[2996]: accepted connection from pid 2991, user nixbot gitlab # [ 229.612993] nix-daemon[2996]: accepted connection from pid 3056, user nixbot gitlab # [ 229.647606] gitaly[893]: time="2026-09-06T16:09:10.248Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP59P21M7RBAB7CZWCCP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:09:40.046" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:10.247" grpc.time_ms=0.643 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.745348] gitaly[893]: time="2026-09-06T16:09:10.345Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQP59P21M7RBAB7CZWCCP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:09:20.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:10.334" grpc.time_ms=11.59 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 229.825375] nixbot[2766]: INFO: ::ffff:127.0.0.1:50882 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 229.918585] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 320 gitlab # [ 229.921069] nixbot[2766]: {"ts": "2026-09-06T16:09:10.519Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 \"HTTP/1.1 201 Created\""} gitlab # [ 229.965120] gitaly[893]: time="2026-09-06T16:09:10.565Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP5KTZDQBP4P9H22DEBAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:09:40.364" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:10.564" grpc.time_ms=0.671 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.997692] gitaly[893]: time="2026-09-06T16:09:10.598Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQP5KTZDQBP4P9H22DEBAD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:09:20.256" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:10.587" grpc.time_ms=11.138 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 230.040747] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 118 gitlab # [ 230.041437] nixbot[2766]: {"ts": "2026-09-06T16:09:10.642Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 230.867302] nixbot[2766]: INFO: ::ffff:127.0.0.1:50884 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 231.904782] nixbot[2766]: INFO: ::ffff:127.0.0.1:50890 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 232.940306] nixbot[2766]: INFO: ::ffff:127.0.0.1:50896 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 233.714311] nix-daemon[2996]: accepted connection from pid 3095, user nixbot gitlab # [ 233.787727] gitaly[893]: time="2026-09-06T16:09:14.387Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP9B28M0NPVDQZFWGVP5A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:09:44.186" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:14.387" grpc.time_ms=0.415 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 233.831983] gitaly[893]: time="2026-09-06T16:09:14.432Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VQP9B28M0NPVDQZFWGVP5A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:09:24.091" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:14.422" grpc.time_ms=10.395 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 233.871679] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 132 gitlab # [ 233.872385] nixbot[2766]: {"ts": "2026-09-06T16:09:14.472Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.02 seconds) gitlab # [ 233.967851] nixbot[2766]: INFO: ::ffff:127.0.0.1:42056 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab # [ 234.190894] gitaly[893]: time="2026-09-06T16:09:14.791Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQP9QKA26CEDJ0SKWRS45Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:14.783" grpc.time_ms=8.125 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.191542] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 52 gitlab # [ 234.269456] gitaly[893]: time="2026-09-06T16:09:14.870Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP9T8JGYCPNNDB0V234CR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:14.861" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 234.269697] gitaly[893]: time="2026-09-06T16:09:14.870Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP9T8JGYCPNNDB0V234CR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:14.861" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.270861] gitaly[893]: time="2026-09-06T16:09:14.870Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP9T8JGYCPNNDB0V234CR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:14.861" grpc.time_ms=9.286 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.330476] gitlab-workhorse[827]: time="2026-09-06T16:09:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 234.330835] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 106 gitlab # [ 234.383918] gitaly[893]: time="2026-09-06T16:09:14.984Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP9XN2HTBFH7PG6X12J7P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:14.972" pid=893 remote_ip="2001:db8:1::1" request_sha=04cc2941e3ad76043a52b5585291b7f01e563985 response_bytes=337 span.kind=server system=grpc gitlab # [ 234.384763] gitaly[893]: time="2026-09-06T16:09:14.984Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP9XN2HTBFH7PG6X12J7P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:14.972" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.410082] gitaly[893]: time="2026-09-06T16:09:15.010Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQP9XN2HTBFH7PG6X12J7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:14.972" grpc.time_ms=12.118 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.410698] gitlab-workhorse[827]: time="2026-09-06T16:09:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 234.411384] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 77 gitlab # [ 234.532292] gitaly[893]: time="2026-09-06T16:09:15.132Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VQPA1W424DAGYVR12RBYRF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:15.121" grpc.time_ms=11.099 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.533065] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 64 gitlab # [ 234.592861] gitaly[893]: time="2026-09-06T16:09:15.193Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQPA3ZS4CXTW6F719JMG44 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:15.185" pid=893 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 234.593359] gitaly[893]: time="2026-09-06T16:09:15.193Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQPA3ZS4CXTW6F719JMG44 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:15.185" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.615438] gitaly[893]: time="2026-09-06T16:09:15.215Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQPA3ZS4CXTW6F719JMG44 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:15.185" grpc.time_ms=8.273 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.615814] gitlab-workhorse[827]: time="2026-09-06T16:09:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 234.615901] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 80 gitlab # [ 234.714584] gitaly[893]: time="2026-09-06T16:09:15.315Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQPA84RW869TWDV30Y753D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:15.309" grpc.time_ms=6.157 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.726675] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 58 gitlab # [ 234.797706] gitaly[893]: time="2026-09-06T16:09:15.397Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQPAA274033PRADGW12JTH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:15.384" pid=893 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 234.798196] gitaly[893]: time="2026-09-06T16:09:15.398Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQPAA274033PRADGW12JTH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:15.384" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.798948] gitlab-workhorse[827]: time="2026-09-06T16:09:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 234.799675] gitaly[893]: time="2026-09-06T16:09:15.398Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQPAA274033PRADGW12JTH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:15.384" grpc.time_ms=13.215 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.838164] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 67 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 234.997221] nixbot[2766]: INFO: ::ffff:127.0.0.1:42066 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 236.032618] nixbot[2766]: INFO: ::ffff:127.0.0.1:42080 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 237.080082] nixbot[2766]: INFO: ::ffff:127.0.0.1:42092 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 238.122060] nixbot[2766]: INFO: ::ffff:127.0.0.1:42096 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 239.169865] nixbot[2766]: INFO: ::ffff:127.0.0.1:42106 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 240.216140] nixbot[2766]: INFO: ::ffff:127.0.0.1:42122 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 241.260810] nixbot[2766]: INFO: ::ffff:127.0.0.1:42134 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 242.288389] nixbot[2766]: INFO: ::ffff:127.0.0.1:42142 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 243.320196] nixbot[2766]: INFO: ::ffff:127.0.0.1:42532 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab # [ 244.208648] sidekiq-cluster[2774]: 2026-09-06T16:09:24.808Z pid=2774 tid=xa6 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2774, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>437555200} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 244.348569] nixbot[2766]: INFO: ::ffff:127.0.0.1:42546 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.02 seconds) gitlab # [ 244.411544] sidekiq-cluster[2774]: W, [2026-09-06T16:09:25.012219 #2774] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 244.503984] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:25.104Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 245.378594] nixbot[2766]: INFO: ::ffff:127.0.0.1:42556 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 246.412963] nixbot[2766]: INFO: ::ffff:127.0.0.1:42558 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab # [ 246.935895] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:27.535Z","message":"GitLab reliable fetch activated!"} gitlab # [ 246.936251] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:27.536Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 247.446716] nixbot[2766]: INFO: ::ffff:127.0.0.1:42562 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 248.472173] nixbot[2766]: INFO: ::ffff:127.0.0.1:42578 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.02 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 249.509560] nixbot[2766]: INFO: ::ffff:127.0.0.1:42592 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 250.548623] nixbot[2766]: INFO: ::ffff:127.0.0.1:42600 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 251.587814] nixbot[2766]: INFO: ::ffff:127.0.0.1:42612 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 252.621939] nixbot[2766]: INFO: ::ffff:127.0.0.1:42614 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 253.660063] nixbot[2766]: INFO: ::ffff:127.0.0.1:45294 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 254.700304] nixbot[2766]: INFO: ::ffff:127.0.0.1:45310 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.07 seconds) gitlab # [ 255.773596] nixbot[2766]: INFO: ::ffff:127.0.0.1:45312 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 256.805104] nixbot[2766]: INFO: ::ffff:127.0.0.1:45326 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 257.836319] nixbot[2766]: INFO: ::ffff:127.0.0.1:45328 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 258.870542] nixbot[2766]: INFO: ::ffff:127.0.0.1:45332 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 259.906137] nixbot[2766]: INFO: ::ffff:127.0.0.1:45348 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 260.944348] nixbot[2766]: INFO: ::ffff:127.0.0.1:45354 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 261.982556] nixbot[2766]: INFO: ::ffff:127.0.0.1:45358 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 263.021071] nixbot[2766]: INFO: ::ffff:127.0.0.1:43032 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 264.059833] nixbot[2766]: INFO: ::ffff:127.0.0.1:43040 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 265.096740] nixbot[2766]: INFO: ::ffff:127.0.0.1:43052 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 266.132888] nixbot[2766]: INFO: ::ffff:127.0.0.1:43060 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 267.174540] nixbot[2766]: INFO: ::ffff:127.0.0.1:43074 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 268.211919] nixbot[2766]: INFO: ::ffff:127.0.0.1:43076 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 269.250322] nixbot[2766]: INFO: ::ffff:127.0.0.1:43086 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab # [ 269.253450] postgres[3316]: [3316] ERROR: relation "table_without_models" does not exist gitlab # [ 269.255200] postgres[3316]: [3316] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 270.293550] nixbot[2766]: INFO: ::ffff:127.0.0.1:43094 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 271.335621] nixbot[2766]: INFO: ::ffff:127.0.0.1:43110 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 271.815851] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:52.416Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 271.816329] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:52.416Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 271.818265] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:52.417Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 271.819375] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:52.417Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 271.820576] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:52.418Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 272.376943] nixbot[2766]: INFO: ::ffff:127.0.0.1:43114 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 273.332829] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:53.932Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d116e2ecc7a77dd939af0f18","created_at":"2026-09-06T16:08:52.756Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:52.783Z","job_size_bytes":2,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d116e2ecc7a77dd939af0f18: start","job_status":"start","queue_duration_s":61.148756,"scheduling_latency_s":61.1488} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 273.411334] nixbot[2766]: INFO: ::ffff:127.0.0.1:40236 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab # [ 273.555738] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.155Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d116e2ecc7a77dd939af0f18","created_at":"2026-09-06T16:08:52.756Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:52.783Z","job_size_bytes":2,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d116e2ecc7a77dd939af0f18: done: 0.223312 sec","job_status":"done","queue_duration_s":61.148756,"scheduling_latency_s":61.1488,"redis_calls":24,"redis_duration_s":0.007115,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.003827,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001962,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001326,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.095200367,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.223312,"completed_at":"2026-09-06T16:09:54.155Z","load_balancing_strategy":"primary","db_duration_s":0.066238,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 273.622471] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.222Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f8905233679e78c64c410079","created_at":"2026-09-06T16:08:52.792Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:52.796Z","job_size_bytes":2,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f8905233679e78c64c410079: start","job_status":"start","queue_duration_s":61.426331,"scheduling_latency_s":61.426361} gitlab # [ 273.749165] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.349Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f8905233679e78c64c410079","created_at":"2026-09-06T16:08:52.792Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:52.796Z","job_size_bytes":2,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f8905233679e78c64c410079: done: 0.126758 sec","job_status":"done","queue_duration_s":61.426331,"scheduling_latency_s":61.426361,"redis_calls":14,"redis_duration_s":0.0037080000000000004,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000866,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001569,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08597461399999999,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.126758,"completed_at":"2026-09-06T16:09:54.349Z","load_balancing_strategy":"primary","db_duration_s":0.022817,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 273.830098] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.430Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"7d7dd337a158a18615bbd64f","created_at":"2026-09-06T16:08:53.266Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:53.271Z","job_size_bytes":5,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-7d7dd337a158a18615bbd64f: start","job_status":"start","queue_duration_s":61.158657,"scheduling_latency_s":61.158688} gitlab # [ 273.962727] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.562Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"7d7dd337a158a18615bbd64f","created_at":"2026-09-06T16:08:53.266Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:53.271Z","job_size_bytes":5,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-7d7dd337a158a18615bbd64f: done: 0.132557 sec","job_status":"done","queue_duration_s":61.158657,"scheduling_latency_s":61.158688,"redis_calls":12,"redis_duration_s":0.003132,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000672,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001651,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000809,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000301435,"duration_s":0.132557,"completed_at":"2026-09-06T16:09:54.562Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026524,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 274.039467] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.639Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"d9b979f9a380b4d94fabe51e","created_at":"2026-09-06T16:08:53.294Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:53.298Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-d9b979f9a380b4d94fabe51e: start","job_status":"start","queue_duration_s":61.341344,"scheduling_latency_s":61.341415} gitlab # [ 274.217286] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.817Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"d9b979f9a380b4d94fabe51e","created_at":"2026-09-06T16:08:53.294Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:53.298Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-d9b979f9a380b4d94fabe51e: done: 0.177575 sec","job_status":"done","queue_duration_s":61.341344,"scheduling_latency_s":61.341415,"redis_calls":11,"redis_duration_s":0.0024460000000000003,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000663,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.123,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.123,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.12122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000393067,"duration_s":0.177575,"completed_at":"2026-09-06T16:09:54.817Z","load_balancing_strategy":"primary","db_duration_s":0.041619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.269832] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.870Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"aab62a7147f33af740703430","created_at":"2026-09-06T16:08:53.301Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:53.304Z","job_size_bytes":89,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-aab62a7147f33af740703430: start","job_status":"start","queue_duration_s":61.565638,"scheduling_latency_s":61.565663} gitlab # [ 274.329791] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.929Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"aab62a7147f33af740703430","created_at":"2026-09-06T16:08:53.301Z","trace_propagation_headers":{"sentry-trace":"a94144ba82de43bf8de197a47268d4fa-d792b63146f74a2c","baggage":"sentry-trace_id=a94144ba82de43bf8de197a47268d4fa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNK364MK1ABHS1VY6CY6M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:08:53.304Z","job_size_bytes":89,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-aab62a7147f33af740703430: done: 0.059447 sec","job_status":"done","queue_duration_s":61.565638,"scheduling_latency_s":61.565663,"redis_calls":11,"redis_duration_s":0.0028669999999999998,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000719,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000526,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00066042,"duration_s":0.059447,"completed_at":"2026-09-06T16:09:54.929Z","load_balancing_strategy":"primary","db_duration_s":0.006137,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.331840] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.932Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 f12cff88c90489b6aedd8947843349145d58ca39 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7e331103800330a73e56087f","created_at":"2026-09-06T16:08:55.312Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-b82bbb3036d24979","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:55.345Z","job_size_bytes":185,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7e331103800330a73e56087f: start","job_status":"start","queue_duration_s":59.587114,"scheduling_latency_s":59.587144} gitlab # [ 274.354801] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:54.954Z","message":"changes: \"0000000000000000000000000000000000000000 f12cff88c90489b6aedd8947843349145d58ca39 refs/heads/master\\n\""} gitlab # [ 274.393073] gitaly[893]: time="2026-09-06T16:09:54.992Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:04.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:54.986" grpc.time_ms=5.873 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 274.410824] gitaly[893]: time="2026-09-06T16:09:55.011Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:10:04.020" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:55.001" grpc.time_ms=10.187 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.439310] nixbot[2766]: INFO: ::ffff:127.0.0.1:40246 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab # [ 274.457946] gitaly[893]: time="2026-09-06T16:09:55.058Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:10:05.010" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:55.050" grpc.time_ms=8.191 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.504305] gitaly[893]: time="2026-09-06T16:09:55.104Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3932 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:55.104" grpc.time_ms=0.488 pid=893 span.kind=server system=grpc gitlab # [ 274.532638] gitaly[893]: time="2026-09-06T16:09:55.133Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3934 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:55.133" grpc.time_ms=0.347 pid=893 span.kind=server system=grpc gitlab # [ 274.548812] gitaly[893]: time="2026-09-06T16:09:55.149Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW diskcache=1eaefadc-302c-436a-b8c7-59770be87626 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:10:05.032" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:55.073" pid=893 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 274.549293] gitaly[893]: time="2026-09-06T16:09:55.149Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=65 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5008 command.oublock=8 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=40 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:10:05.032" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:55.073" grpc.time_ms=76.536 pid=893 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 274.566879] gitaly[893]: time="2026-09-06T16:09:55.167Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:10:05.127" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:55.167" grpc.time_ms=0.123 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.580924] gitaly[893]: time="2026-09-06T16:09:55.181Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=944 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:10:05.129" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:55.169" grpc.time_ms=11.581 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.646553] gitaly[893]: time="2026-09-06T16:09:55.247Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-06T16:10:25.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:55.239" grpc.time_ms=7.489 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.677827] gitaly[893]: time="2026-09-06T16:09:55.278Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:10:05.237" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:55.278" grpc.time_ms=0.189 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.687072] gitaly[893]: time="2026-09-06T16:09:55.287Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:10:05.239" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:09:55.279" grpc.time_ms=7.885 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.700888] gitaly[893]: time="2026-09-06T16:09:55.301Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:05.256" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:55.296" grpc.time_ms=4.924 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.712937] gitaly[893]: time="2026-09-06T16:09:55.313Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:05.264" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:55.305" grpc.time_ms=8.335 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.730924] gitaly[893]: time="2026-09-06T16:09:55.331Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:10:25.120" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:55.321" grpc.time_ms=10.226 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.078762] gitaly[893]: time="2026-09-06T16:09:55.679Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:10:05.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:55.678" grpc.time_ms=1.196 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.124886] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:55.724Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 f12cff88c90489b6aedd8947843349145d58ca39 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7e331103800330a73e56087f","created_at":"2026-09-06T16:08:55.312Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-b82bbb3036d24979","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:55.345Z","job_size_bytes":185,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7e331103800330a73e56087f: done: 0.792539 sec","job_status":"done","queue_duration_s":59.587114,"scheduling_latency_s":59.587144,"gitaly_calls":13,"gitaly_duration_s":0.1971,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.046073,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.003588,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.00774,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001622,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003712,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.018738,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.010673,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.173,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.193,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.477681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000706515,"exclusive_lock_hold_duration_s":0.025611076,"duration_s":0.792539,"completed_at":"2026-09-06T16:09:55.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.092629,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 275.200795] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:55.727Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","f12cff88c90489b6aedd8947843349145d58ca39 f684670ecafb28f8f83038a891201ff920ef7429 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"57b167904a1fe8879a26333e","created_at":"2026-09-06T16:08:59.834Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-ef0e5423b75b4d7e","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:59.836Z","job_size_bytes":185,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-57b167904a1fe8879a26333e: start","job_status":"start","queue_duration_s":55.891355,"scheduling_latency_s":55.891391} gitlab # [ 275.201754] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:55.744Z","message":"changes: \"f12cff88c90489b6aedd8947843349145d58ca39 f684670ecafb28f8f83038a891201ff920ef7429 refs/heads/master\\n\""} gitlab # [ 275.202603] gitaly[893]: time="2026-09-06T16:09:55.772Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:05.094" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:55.764" grpc.time_ms=8.023 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.243489] gitaly[893]: time="2026-09-06T16:09:55.844Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:10:05.168" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:55.838" grpc.time_ms=5.387 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.249872] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:55.850Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0dbdeee8ceb1c20846643b82","created_at":"2026-09-06T16:09:55.847Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bce5b304b0fa4e42","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"c9f2d30319cdeb5ad3137d79","job_size_bytes":46,"pid":2774,"job_status":"deduplicated","message":"ProjectCacheWorker JID-0dbdeee8ceb1c20846643b82: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 275.270466] gitaly[893]: time="2026-09-06T16:09:55.871Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:10:05.193" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:55.864" grpc.time_ms=6.989 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.280273] gitaly[893]: time="2026-09-06T16:09:55.880Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-06T16:10:25.675" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:55.875" grpc.time_ms=5.59 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.311439] gitaly[893]: time="2026-09-06T16:09:55.912Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:10:05.237" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:09:55.907" grpc.time_ms=4.893 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.331049] gitaly[893]: time="2026-09-06T16:09:55.931Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:05.252" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:55.922" grpc.time_ms=9.292 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.397402] gitaly[893]: time="2026-09-06T16:09:55.997Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:10:05.016" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:55.996" grpc.time_ms=0.677 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 275.429596] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:56.029Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","f12cff88c90489b6aedd8947843349145d58ca39 f684670ecafb28f8f83038a891201ff920ef7429 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"57b167904a1fe8879a26333e","created_at":"2026-09-06T16:08:59.834Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-ef0e5423b75b4d7e","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:08:59.836Z","job_size_bytes":185,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-57b167904a1fe8879a26333e: done: 0.301974 sec","job_status":"done","queue_duration_s":55.891355,"scheduling_latency_s":55.891391,"gitaly_calls":7,"gitaly_duration_s":0.132545,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.037543,"redis_read_bytes":500,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.001764,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000841,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003204,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007656,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.024078,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.018,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.018,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.12666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00074367,"exclusive_lock_hold_duration_s":0.079143858,"duration_s":0.301974,"completed_at":"2026-09-06T16:09:56.029Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021992,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 275.476228] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:56.076Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f816c317b8ba8aa3ec744c47","created_at":"2026-09-06T16:09:00.259Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-089cad70a2294b4e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:00.263Z","job_size_bytes":65,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f816c317b8ba8aa3ec744c47: start","job_status":"start","queue_duration_s":55.812477,"scheduling_latency_s":55.812506} gitlab # [ 275.516274] nixbot[2766]: INFO: ::ffff:127.0.0.1:40254 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.08 seconds) gitlab # [ 275.585577] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:56.185Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f816c317b8ba8aa3ec744c47","created_at":"2026-09-06T16:09:00.259Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-089cad70a2294b4e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:00.263Z","job_size_bytes":65,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f816c317b8ba8aa3ec744c47: done: 0.109053 sec","job_status":"done","queue_duration_s":55.812477,"scheduling_latency_s":55.812506,"redis_calls":16,"redis_duration_s":0.004543,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002231,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00061,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000443352,"duration_s":0.109053,"completed_at":"2026-09-06T16:09:56.185Z","load_balancing_strategy":"primary","db_duration_s":0.023315,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 275.661790] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:56.261Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ffde054c3f7df7bacb0ac793","created_at":"2026-09-06T16:09:00.287Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-089cad70a2294b4e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:00.291Z","job_size_bytes":65,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ffde054c3f7df7bacb0ac793: start","job_status":"start","queue_duration_s":55.970016,"scheduling_latency_s":55.97006} gitlab # [ 275.841073] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:56.440Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ffde054c3f7df7bacb0ac793","created_at":"2026-09-06T16:09:00.287Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-089cad70a2294b4e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:00.291Z","job_size_bytes":65,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ffde054c3f7df7bacb0ac793: done: 0.179258 sec","job_status":"done","queue_duration_s":55.970016,"scheduling_latency_s":55.97006,"redis_calls":21,"redis_duration_s":0.0064210000000000005,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002723,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001917,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001781,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.132716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000551467,"duration_s":0.179258,"completed_at":"2026-09-06T16:09:56.440Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.843591] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:56.442Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e94310c24f5e54d896279df6","created_at":"2026-09-06T16:09:00.362Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-089cad70a2294b4e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:00.366Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e94310c24f5e54d896279df6: start","job_status":"start","queue_duration_s":56.076429,"scheduling_latency_s":56.076459} gitlab # [ 276.065951] gitaly[893]: time="2026-09-06T16:09:56.666Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:56.665" grpc.time_ms=0.614 pid=893 span.kind=server system=grpc gitlab # [ 276.118588] gitaly[893]: time="2026-09-06T16:09:56.719Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:56.718" grpc.time_ms=0.253 pid=893 span.kind=server system=grpc gitlab # [ 276.179611] gitaly[893]: time="2026-09-06T16:09:56.780Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:56.779" grpc.time_ms=0.583 pid=893 span.kind=server system=grpc gitlab # [ 276.214481] gitaly[893]: time="2026-09-06T16:09:56.815Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y diskcache=ec36c930-dcfc-4db7-9e03-842483bb7b7b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:26.425" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:56.626" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.229474] gitaly[893]: time="2026-09-06T16:09:56.815Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=185 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7524 command.oublock=0 command.real_time_ms=181 command.spawn_token_fork_ms=0 command.system_time_ms=127 command.user_time_ms=58 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:26.425" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:56.626" grpc.time_ms=188.843 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.383591] gitaly[893]: time="2026-09-06T16:09:56.984Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:10:26.782" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:56.982" grpc.time_ms=1.296 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.435401] gitaly[893]: time="2026-09-06T16:09:57.035Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:27.033" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.034" grpc.time_ms=0.959 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.480790] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.080Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e94310c24f5e54d896279df6","created_at":"2026-09-06T16:09:00.362Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-089cad70a2294b4e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:00.366Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2774,"job_status":"rescheduled","message":"PipelineProcessWorker JID-e94310c24f5e54d896279df6: rescheduled"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 276.494075] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.094Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e94310c24f5e54d896279df6","created_at":"2026-09-06T16:09:00.362Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-089cad70a2294b4e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:00.366Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e94310c24f5e54d896279df6: done: 0.651294 sec","job_status":"done","queue_duration_s":56.076429,"scheduling_latency_s":56.076459,"gitaly_calls":3,"gitaly_duration_s":0.200761,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.06662,"redis_read_bytes":462,"redis_write_bytes":29294,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001385,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002687,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.014603,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.006344,"redis_queues_read_bytes":39,"redis_queues_write_bytes":15582,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.007776,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000818,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000731,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.030891,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.025,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.047,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.282998,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.539201769,"duration_s":0.651294,"completed_at":"2026-09-06T16:09:57.094Z","load_balancing_strategy":"primary","db_duration_s":0.096258,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 276.562270] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.096Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c9f2d30319cdeb5ad3137d79","created_at":"2026-09-06T16:09:55.013Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.028Z","job_size_bytes":46,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c9f2d30319cdeb5ad3137d79: start","job_status":"start","queue_duration_s":2.067398,"scheduling_latency_s":2.06743} gitlab # [ 276.562899] gitaly[893]: time="2026-09-06T16:09:57.151Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-06T22:09:57.145" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:57.145" grpc.time_ms=5.034 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.563544] gitaly[893]: time="2026-09-06T16:09:57.162Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:10:27.156" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.156" grpc.time_ms=5.957 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.10 seconds) gitlab # [ 276.597362] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.197Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c9f2d30319cdeb5ad3137d79","created_at":"2026-09-06T16:09:55.013Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.028Z","job_size_bytes":46,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c9f2d30319cdeb5ad3137d79: done: 0.101155 sec","job_status":"done","queue_duration_s":2.067398,"scheduling_latency_s":2.06743,"gitaly_calls":2,"gitaly_duration_s":0.015564,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008181,"redis_read_bytes":69,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001783,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000596,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001845,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002976,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000458717,"duration_s":0.101155,"completed_at":"2026-09-06T16:09:57.197Z","load_balancing_strategy":"primary","db_duration_s":0.014312,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 276.607079] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.199Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"e6cca2d5dd58abeecbef0782","created_at":"2026-09-06T16:09:55.043Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.047Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-e6cca2d5dd58abeecbef0782: start","job_status":"start","queue_duration_s":2.152666,"scheduling_latency_s":2.152697} gitlab # [ 276.648072] nixbot[2766]: INFO: ::ffff:127.0.0.1:40258 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab # [ 276.648708] gitaly[893]: time="2026-09-06T16:09:57.248Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW diskcache=63df11dc-83f5-45f9-9f88-f1cc05706107 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:09:57.226" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.227" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.649520] gitaly[893]: time="2026-09-06T16:09:57.248Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1289 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:09:57.226" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.227" grpc.time_ms=20.91 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.688902] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.289Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"e6cca2d5dd58abeecbef0782","created_at":"2026-09-06T16:09:55.043Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.047Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-e6cca2d5dd58abeecbef0782: done: 0.089366 sec","job_status":"done","queue_duration_s":2.152666,"scheduling_latency_s":2.152697,"gitaly_calls":1,"gitaly_duration_s":0.024334,"redis_calls":15,"redis_duration_s":0.004046,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000598,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.019,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.058877874,"duration_s":0.089366,"completed_at":"2026-09-06T16:09:57.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012951,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.690721] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.291Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2cf27eaeaadba2d6a269b52f","created_at":"2026-09-06T16:09:55.163Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.165Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2cf27eaeaadba2d6a269b52f: start","job_status":"start","queue_duration_s":2.125985,"scheduling_latency_s":2.126016} gitlab # [ 276.725080] gitaly[893]: time="2026-09-06T16:09:57.325Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:10:27.119" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.320" grpc.time_ms=5.592 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.731504] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.332Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2cf27eaeaadba2d6a269b52f","created_at":"2026-09-06T16:09:55.163Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.165Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2cf27eaeaadba2d6a269b52f: done: 0.04094 sec","job_status":"done","queue_duration_s":2.125985,"scheduling_latency_s":2.126016,"gitaly_calls":1,"gitaly_duration_s":0.00806,"redis_calls":16,"redis_duration_s":0.003146,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000171,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001408,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04094,"completed_at":"2026-09-06T16:09:57.332Z","load_balancing_strategy":"primary","db_duration_s":0.002921,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 276.755755] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:57.333Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"573f8e7128116ced51faa8e4","created_at":"2026-09-06T16:09:55.560Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c33f7e71881805941e524ef2d3021e5dad56ff9032048f2850e2e230c756daf2","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.564Z","job_size_bytes":467,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-573f8e7128116ced51faa8e4: start","job_status":"start","queue_duration_s":1.768862,"scheduling_latency_s":1.768878} gitlab # [ 276.806495] gitaly[893]: time="2026-09-06T16:09:57.351Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:27.149" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.350" grpc.time_ms=0.523 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.807242] gitaly[893]: time="2026-09-06T16:09:57.360Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:10:07.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:57.353" grpc.time_ms=6.686 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.838224] gitaly[893]: time="2026-09-06T16:09:57.438Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:10:27.238" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.438" grpc.time_ms=0.637 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.843473] gitaly[893]: time="2026-09-06T16:09:57.444Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=28 command.oublock=0 command.real_time_ms=0 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:10:27.242" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.442" grpc.time_ms=1.357 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.904819] gitaly[893]: time="2026-09-06T16:09:57.505Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:07.146" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:57.496" grpc.time_ms=8.535 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.360941] gitaly[893]: time="2026-09-06T16:09:57.961Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:10:27.743" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.944" grpc.time_ms=17.184 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.411409] gitaly[893]: time="2026-09-06T16:09:58.012Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNNRRN45T1NSMNR97ZGDW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:10:27.797" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:57.997" grpc.time_ms=14.421 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.469901] sidekiq-cluster[2774]: {"severity":"WARN","time":"2026-09-06T16:09:58.070Z","class":"CreatePipelineWorker","correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"f12cff88c90489b6aedd8947843349145d58ca39","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"f12cff88c90489b6aedd8947843349145d58ca39","ref":"refs/heads/master"},"retry":0} gitlab # [ 277.481806] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.081Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"573f8e7128116ced51faa8e4","created_at":"2026-09-06T16:09:55.560Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c33f7e71881805941e524ef2d3021e5dad56ff9032048f2850e2e230c756daf2","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.564Z","job_size_bytes":467,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-573f8e7128116ced51faa8e4: done: 0.747869 sec","job_status":"done","queue_duration_s":1.768862,"scheduling_latency_s":1.768878,"gitaly_calls":7,"gitaly_duration_s":0.121668,"redis_calls":51,"redis_duration_s":0.010554000000000001,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000713,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004034,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001569,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003014,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.57759,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000586108,"duration_s":0.747869,"completed_at":"2026-09-06T16:09:58.081Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035399,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 277.546103] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.083Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"ada602a0-fb1a-4ccd-8134-64e739f50059\"}"],"class":"WebHookWorker","jid":"30f075a1224d747af0c2571c","created_at":"2026-09-06T16:09:55.653Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd6cfd137e7b3c177592f8929bf83d43ce63358d12a5f4a6108464d82fc6762e","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.658Z","job_size_bytes":1751,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-30f075a1224d747af0c2571c: start","job_status":"start","queue_duration_s":2.425157,"scheduling_latency_s":2.425182} gitlab # [ 277.602690] nixbot[2766]: INFO: ::ffff:127.0.0.1:40274 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.04 seconds) gitlab # [ 277.621286] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.221Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"ada602a0-fb1a-4ccd-8134-64e739f50059\"}"],"class":"WebHookWorker","jid":"30f075a1224d747af0c2571c","created_at":"2026-09-06T16:09:55.653Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-7471dcf155174cdb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd6cfd137e7b3c177592f8929bf83d43ce63358d12a5f4a6108464d82fc6762e","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.658Z","job_size_bytes":1751,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-30f075a1224d747af0c2571c: done: 0.137646 sec","job_status":"done","queue_duration_s":2.425157,"scheduling_latency_s":2.425182,"redis_calls":19,"redis_duration_s":0.004413,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001182,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000376,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001921,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0546692390000203,"cpu_s":0.06724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000358705,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.137646,"completed_at":"2026-09-06T16:09:58.221Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.628340] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.226Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"d9a5170316904cc4e9f44ba8","created_at":"2026-09-06T16:09:55.857Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bce5b304b0fa4e42","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"e6cca2d5dd58abeecbef0782","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.859Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-d9a5170316904cc4e9f44ba8: start","job_status":"start","queue_duration_s":2.366513,"scheduling_latency_s":2.366542} gitlab # [ 277.658256] nixbot[2766]: INFO: ::ffff:127.0.0.1:40284 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab # [ 277.693923] gitaly[893]: time="2026-09-06T16:09:58.294Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 diskcache=d6ec4728-12de-4c52-8b0c-cd1aa7665a9c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:09:58.243" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.243" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.694277] gitaly[893]: time="2026-09-06T16:09:58.294Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=45 command.spawn_token_fork_ms=43 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:09:58.243" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.243" grpc.time_ms=50.839 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.710098] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.310Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"d9a5170316904cc4e9f44ba8","created_at":"2026-09-06T16:09:55.857Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bce5b304b0fa4e42","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"e6cca2d5dd58abeecbef0782","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.859Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-d9a5170316904cc4e9f44ba8: done: 0.084071 sec","job_status":"done","queue_duration_s":2.366513,"scheduling_latency_s":2.366542,"gitaly_calls":1,"gitaly_duration_s":0.053619,"redis_calls":12,"redis_duration_s":0.0021739999999999997,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000303,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.065788529,"duration_s":0.084071,"completed_at":"2026-09-06T16:09:58.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003578,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.751073] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.314Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"dbf613bbf87ee6bf056b62b5","created_at":"2026-09-06T16:09:55.975Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bce5b304b0fa4e42","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:67f7758958e180ab78f51789aa5eeed87bc540b35c9a504bfc7e39c0b1d6efae","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.977Z","job_size_bytes":467,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-dbf613bbf87ee6bf056b62b5: start","job_status":"start","queue_duration_s":2.336595,"scheduling_latency_s":2.33661} gitlab # [ 277.752277] gitaly[893]: time="2026-09-06T16:09:58.330Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:28.129" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.329" grpc.time_ms=0.471 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.753252] gitaly[893]: time="2026-09-06T16:09:58.337Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:10:08.001" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:58.331" grpc.time_ms=5.532 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.759620] gitaly[893]: time="2026-09-06T16:09:58.360Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:10:28.359" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.359" grpc.time_ms=0.633 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.764912] gitaly[893]: time="2026-09-06T16:09:58.365Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=199 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:10:28.362" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.362" grpc.time_ms=2.951 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.812076] gitaly[893]: time="2026-09-06T16:09:58.371Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQM8GMBTSGD9BF57CW4RG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:58.355" grpc.time_ms=15.731 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 277.813054] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:58 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 84 gitlab # [ 277.813554] gitaly[893]: time="2026-09-06T16:09:58.391Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:08.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:58.382" grpc.time_ms=8.481 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.873837] gitaly[893]: time="2026-09-06T16:09:58.474Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQMB7PFVE88X1QNYDT9YJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:58.467" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 277.874184] gitaly[893]: time="2026-09-06T16:09:58.474Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQMB7PFVE88X1QNYDT9YJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:58.467" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 277.875114] gitaly[893]: time="2026-09-06T16:09:58.474Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQMB7PFVE88X1QNYDT9YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:58.467" grpc.time_ms=7.667 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 277.875960] gitlab-workhorse[827]: time="2026-09-06T16:09:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 277.876581] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:58 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 99 gitlab # [ 277.928420] nixbot[2766]: {"ts": "2026-09-06T16:09:58.529Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 277.978886] gitaly[893]: time="2026-09-06T16:09:58.579Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQMGB11410J7M16ZH0NY4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:28.378" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.578" grpc.time_ms=0.691 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.040060] gitaly[893]: time="2026-09-06T16:09:58.640Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQMGB11410J7M16ZH0NY4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:08.300" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:58.631" grpc.time_ms=9.573 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.094712] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 155 gitlab # [ 278.100478] nixbot[2766]: {"ts": "2026-09-06T16:09:58.696Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 \"HTTP/1.1 201 Created\""} gitlab # [ 278.159852] gitaly[893]: time="2026-09-06T16:09:58.760Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=969 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:10:28.747" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.748" grpc.time_ms=12.133 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.165888] gitaly[893]: time="2026-09-06T16:09:58.766Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQMNK7439CG73DTC9XKJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:28.765" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.765" grpc.time_ms=0.497 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.223542] gitaly[893]: time="2026-09-06T16:09:58.824Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNTRZ0AACKBRYAJ2PZ193 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:10:28.814" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:58.814" grpc.time_ms=9.577 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.247749] sidekiq-cluster[2774]: {"severity":"WARN","time":"2026-09-06T16:09:58.848Z","class":"CreatePipelineWorker","correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"f12cff88c90489b6aedd8947843349145d58ca39","after":"f684670ecafb28f8f83038a891201ff920ef7429","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"f684670ecafb28f8f83038a891201ff920ef7429","ref":"refs/heads/master"},"retry":0} gitlab # [ 278.253829] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.854Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"dbf613bbf87ee6bf056b62b5","created_at":"2026-09-06T16:09:55.975Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bce5b304b0fa4e42","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:67f7758958e180ab78f51789aa5eeed87bc540b35c9a504bfc7e39c0b1d6efae","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.977Z","job_size_bytes":467,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-dbf613bbf87ee6bf056b62b5: done: 0.539867 sec","job_status":"done","queue_duration_s":2.336595,"scheduling_latency_s":2.33661,"gitaly_calls":7,"gitaly_duration_s":0.11736,"redis_calls":33,"redis_duration_s":0.0045579999999999996,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000883,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001892,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00081,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.454965,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.539867,"completed_at":"2026-09-06T16:09:58.854Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012459,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 278.273141] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.873Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"1340362e-1607-434c-9cf3-2789538e19f9\"}"],"class":"WebHookWorker","jid":"be9fc1402c340121769713b7","created_at":"2026-09-06T16:09:55.987Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bce5b304b0fa4e42","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c7ce08ec20bbc2bc4eb84473656e43f44e20b41f295556d2572a8cd4a562340","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.988Z","job_size_bytes":1749,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-be9fc1402c340121769713b7: start","job_status":"start","queue_duration_s":2.885222,"scheduling_latency_s":2.88525} gitlab # [ 278.305563] nixbot[2766]: INFO: ::ffff:127.0.0.1:40298 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 278.316682] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.916Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"1340362e-1607-434c-9cf3-2789538e19f9\"}"],"class":"WebHookWorker","jid":"be9fc1402c340121769713b7","created_at":"2026-09-06T16:09:55.987Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bce5b304b0fa4e42","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c7ce08ec20bbc2bc4eb84473656e43f44e20b41f295556d2572a8cd4a562340","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:55.988Z","job_size_bytes":1749,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-be9fc1402c340121769713b7: done: 0.043528 sec","job_status":"done","queue_duration_s":2.885222,"scheduling_latency_s":2.88525,"redis_calls":14,"redis_duration_s":0.002499,"redis_read_bytes":26,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000318,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001309,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.00701681399999643,"cpu_s":0.026213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.043528,"completed_at":"2026-09-06T16:09:58.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.359669] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:58.960Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b01db6cde5b6e93b131da7ce","created_at":"2026-09-06T16:09:56.604Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.611Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b01db6cde5b6e93b131da7ce: start","job_status":"start","queue_duration_s":2.348996,"scheduling_latency_s":2.349022} gitlab # [ 278.373349] gitaly[893]: time="2026-09-06T16:09:58.974Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQMWBSVANHKAHEWDW1M9Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:58.965" grpc.time_ms=8.746 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 278.373968] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:58 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 50 gitlab # [ 278.412919] gitaly[893]: time="2026-09-06T16:09:59.013Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQMNK7439CG73DTC9XKJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:09.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:59.007" grpc.time_ms=5.747 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.418797] gitaly[893]: time="2026-09-06T16:09:59.019Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQMY0E1T0HD6XG6H29A4Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:59.011" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 278.419391] gitaly[893]: time="2026-09-06T16:09:59.019Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQMY0E1T0HD6XG6H29A4Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:59.011" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 278.420069] gitlab-workhorse[827]: time="2026-09-06T16:09:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 278.420502] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 43 gitlab # [ 278.420993] gitaly[893]: time="2026-09-06T16:09:59.019Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQMY0E1T0HD6XG6H29A4Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:59.011" grpc.time_ms=8.131 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 278.453611] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 346 gitlab # [ 278.455135] nixbot[2766]: {"ts": "2026-09-06T16:09:59.055Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 \"HTTP/1.1 201 Created\""} gitlab # [ 278.497591] gitaly[893]: time="2026-09-06T16:09:59.098Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN0KSFV3FDWW57CVR7WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:29.097" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:59.097" grpc.time_ms=0.537 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.505682] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.105Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b01db6cde5b6e93b131da7ce","created_at":"2026-09-06T16:09:56.604Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.611Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b01db6cde5b6e93b131da7ce: done: 0.145172 sec","job_status":"done","queue_duration_s":2.348996,"scheduling_latency_s":2.349022,"redis_calls":10,"redis_duration_s":0.0037560000000000002,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.06,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00264475,"duration_s":0.145172,"completed_at":"2026-09-06T16:09:59.105Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06563,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 278.509252] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.106Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d702f05008b183690f6a8c79","created_at":"2026-09-06T16:09:56.822Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.829Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d702f05008b183690f6a8c79: start","job_status":"start","queue_duration_s":2.276968,"scheduling_latency_s":2.276983} gitlab # [ 278.559893] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.132Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d702f05008b183690f6a8c79","created_at":"2026-09-06T16:09:56.822Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.829Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d702f05008b183690f6a8c79: done: 0.025672 sec","job_status":"done","queue_duration_s":2.276968,"scheduling_latency_s":2.276983,"redis_calls":10,"redis_duration_s":0.0038540000000000002,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000829,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000475,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195276,"duration_s":0.025672,"completed_at":"2026-09-06T16:09:59.132Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003205,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.562214] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"784a8117146376d1c6d105a6","created_at":"2026-09-06T16:09:56.856Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.862Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-784a8117146376d1c6d105a6: start","job_status":"start","queue_duration_s":2.270529,"scheduling_latency_s":2.270541} gitlab # [ 278.576518] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 63 gitlab # [ 278.615570] gitaly[893]: time="2026-09-06T16:09:59.155Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQN1MZ9C74DFX11SPGP7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:59.144" grpc.time_ms=11.372 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 278.616174] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.156Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"784a8117146376d1c6d105a6","created_at":"2026-09-06T16:09:56.856Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.862Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-784a8117146376d1c6d105a6: done: 0.02369 sec","job_status":"done","queue_duration_s":2.270529,"scheduling_latency_s":2.270541,"redis_calls":10,"redis_duration_s":0.001577,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000231,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000301155,"duration_s":0.02369,"completed_at":"2026-09-06T16:09:59.156Z","load_balancing_strategy":"primary","db_duration_s":0.004452,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.627841] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.157Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"da85a55da948ae0e35297f44","created_at":"2026-09-06T16:09:56.880Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.887Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-da85a55da948ae0e35297f44: start","job_status":"start","queue_duration_s":2.270274,"scheduling_latency_s":2.270287} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 0.03 seconds) gitlab # [ 278.641790] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"da85a55da948ae0e35297f44","created_at":"2026-09-06T16:09:56.880Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.887Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-da85a55da948ae0e35297f44: done: 0.02797 sec","job_status":"done","queue_duration_s":2.270274,"scheduling_latency_s":2.270287,"redis_calls":13,"redis_duration_s":0.006583,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001022,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000449,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203099,"duration_s":0.02797,"completed_at":"2026-09-06T16:09:59.185Z","load_balancing_strategy":"primary","db_duration_s":0.002996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.694362] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.186Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"60b50f18df9856f1b84379f5","created_at":"2026-09-06T16:09:56.888Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.895Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-60b50f18df9856f1b84379f5: start","job_status":"start","queue_duration_s":2.290868,"scheduling_latency_s":2.290881} gitlab # [ 278.695305] gitaly[893]: time="2026-09-06T16:09:59.229Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN4BJP3HP0NFJ8RJJGR4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:59.219" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 278.703079] gitlab-workhorse[827]: time="2026-09-06T16:09:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 278.704928] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 51 gitlab # [ 278.706671] nixbot[2766]: INFO: ::ffff:127.0.0.1:40302 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab # [ 278.708059] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.209Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"60b50f18df9856f1b84379f5","created_at":"2026-09-06T16:09:56.888Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.895Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-60b50f18df9856f1b84379f5: done: 0.022951 sec","job_status":"done","queue_duration_s":2.290868,"scheduling_latency_s":2.290881,"redis_calls":10,"redis_duration_s":0.001864,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000274,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208965,"duration_s":0.022951,"completed_at":"2026-09-06T16:09:59.209Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002624,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.773976] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.210Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1dcedac245e8a27f8abd2606","created_at":"2026-09-06T16:09:56.896Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.903Z","job_size_bytes":19,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1dcedac245e8a27f8abd2606: start","job_status":"start","queue_duration_s":2.307106,"scheduling_latency_s":2.307119} gitlab # [ 278.774966] gitaly[893]: time="2026-09-06T16:09:59.229Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN4BJP3HP0NFJ8RJJGR4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:59.219" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 278.775906] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1dcedac245e8a27f8abd2606","created_at":"2026-09-06T16:09:56.896Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.903Z","job_size_bytes":19,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1dcedac245e8a27f8abd2606: done: 0.02843 sec","job_status":"done","queue_duration_s":2.307106,"scheduling_latency_s":2.307119,"redis_calls":11,"redis_duration_s":0.008473,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002862,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004824,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000787,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001075276,"duration_s":0.02843,"completed_at":"2026-09-06T16:09:59.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.817576] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.242Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"cc8f4ab6f6c33a0994dabae5","created_at":"2026-09-06T16:09:56.907Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.915Z","job_size_bytes":66,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-cc8f4ab6f6c33a0994dabae5: start","job_status":"start","queue_duration_s":2.326895,"scheduling_latency_s":2.326909} gitlab # [ 278.826894] gitaly[893]: time="2026-09-06T16:09:59.229Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN4BJP3HP0NFJ8RJJGR4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:09:59.219" grpc.time_ms=10.065 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 278.866701] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.355Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"cc8f4ab6f6c33a0994dabae5","created_at":"2026-09-06T16:09:56.907Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.915Z","job_size_bytes":66,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-cc8f4ab6f6c33a0994dabae5: done: 0.113856 sec","job_status":"done","queue_duration_s":2.326895,"scheduling_latency_s":2.326909,"redis_calls":10,"redis_duration_s":0.003985,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000897,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002624,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.089,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.087,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375188,"duration_s":0.113856,"completed_at":"2026-09-06T16:09:59.355Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.089786,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.879591] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.357Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7d1377289e54aed8339b577c","created_at":"2026-09-06T16:09:56.916Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.922Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7d1377289e54aed8339b577c: start","job_status":"start","queue_duration_s":2.434465,"scheduling_latency_s":2.434479} gitlab # [ 278.924709] gitaly[893]: time="2026-09-06T16:09:59.417Z" level=info msg="finished unary call with code OK" catfile.duration_ms=21 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=21 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:29.394" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:59.394" grpc.time_ms=21.615 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.925531] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.383Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7d1377289e54aed8339b577c","created_at":"2026-09-06T16:09:56.916Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.922Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7d1377289e54aed8339b577c: done: 0.026804 sec","job_status":"done","queue_duration_s":2.434465,"scheduling_latency_s":2.434479,"redis_calls":10,"redis_duration_s":0.001293,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197231,"duration_s":0.026804,"completed_at":"2026-09-06T16:09:59.383Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.944033] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.384Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ab1330143d7aff70ef3b5d81","created_at":"2026-09-06T16:09:56.933Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.939Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ab1330143d7aff70ef3b5d81: start","job_status":"start","queue_duration_s":2.445613,"scheduling_latency_s":2.445625} gitlab # [ 278.993422] gitaly[893]: time="2026-09-06T16:09:59.429Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:10:29.423" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:59.423" grpc.time_ms=5.954 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 278.994046] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.408Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ab1330143d7aff70ef3b5d81","created_at":"2026-09-06T16:09:56.933Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.939Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ab1330143d7aff70ef3b5d81: done: 0.02331 sec","job_status":"done","queue_duration_s":2.445613,"scheduling_latency_s":2.445625,"redis_calls":10,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000743,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00019388,"duration_s":0.02331,"completed_at":"2026-09-06T16:09:59.408Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.042915] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.409Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"52ca93afe3073a22310ccf22","created_at":"2026-09-06T16:09:56.940Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.946Z","job_size_bytes":6,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-52ca93afe3073a22310ccf22: start","job_status":"start","queue_duration_s":2.463011,"scheduling_latency_s":2.463024} gitlab # [ 279.043629] gitaly[893]: time="2026-09-06T16:09:59.507Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:09.160" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:59.501" grpc.time_ms=6.696 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.044245] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.429Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"52ca93afe3073a22310ccf22","created_at":"2026-09-06T16:09:56.940Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.946Z","job_size_bytes":6,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-52ca93afe3073a22310ccf22: done: 0.020945 sec","job_status":"done","queue_duration_s":2.463011,"scheduling_latency_s":2.463024,"redis_calls":10,"redis_duration_s":0.0051909999999999994,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000234,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001472,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001284521,"duration_s":0.020945,"completed_at":"2026-09-06T16:09:59.429Z","load_balancing_strategy":"primary","db_duration_s":0.001265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.095814] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.431Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"bc4fb2a40f448238ecc29119","created_at":"2026-09-06T16:09:56.946Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.952Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bc4fb2a40f448238ecc29119: start","job_status":"start","queue_duration_s":2.479149,"scheduling_latency_s":2.479169} gitlab # [ 279.096685] gitaly[893]: time="2026-09-06T16:09:59.584Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:10:09.244" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:59.584" grpc.time_ms=0.212 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.098883] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.585Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"bc4fb2a40f448238ecc29119","created_at":"2026-09-06T16:09:56.946Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.952Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bc4fb2a40f448238ecc29119: done: 0.153801 sec","job_status":"done","queue_duration_s":2.479149,"scheduling_latency_s":2.479169,"redis_calls":10,"redis_duration_s":0.001605,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000356,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000237181,"duration_s":0.153801,"completed_at":"2026-09-06T16:09:59.585Z","load_balancing_strategy":"primary","db_duration_s":0.064826,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.167587] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.586Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a76e656367b504465ad6e334","created_at":"2026-09-06T16:09:56.961Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.967Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a76e656367b504465ad6e334: start","job_status":"start","queue_duration_s":2.619129,"scheduling_latency_s":2.619144} gitlab # [ 279.168222] gitaly[893]: time="2026-09-06T16:09:59.592Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:10:09.246" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:59.587" grpc.time_ms=4.856 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.168802] gitaly[893]: time="2026-09-06T16:09:59.626Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4149 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:59.626" grpc.time_ms=0.338 pid=893 span.kind=server system=grpc gitlab # [ 279.171758] nixbot[2766]: {"ts": "2026-09-06T16:09:59.722Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f684670ecafb28f8f83038a891201ff920ef7429 \"HTTP/1.1 201 Created\""} gitlab # [ 279.213738] nixbot[2766]: {"ts": "2026-09-06T16:09:59.732Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-f684670ecafb-5"} gitlab # [ 279.214547] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.634Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a76e656367b504465ad6e334","created_at":"2026-09-06T16:09:56.961Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:56.967Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a76e656367b504465ad6e334: done: 0.048012 sec","job_status":"done","queue_duration_s":2.619129,"scheduling_latency_s":2.619144,"redis_calls":14,"redis_duration_s":0.004894,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001781,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002298,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239415,"duration_s":0.048012,"completed_at":"2026-09-06T16:09:59.634Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010341,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.215843] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.635Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f3077871eb8eeb032d39d814","created_at":"2026-09-06T16:09:57.081Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:57.083Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f3077871eb8eeb032d39d814: start","job_status":"start","queue_duration_s":2.552254,"scheduling_latency_s":2.552291} gitlab # [ 279.230430] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 362 gitlab # [ 279.232544] gitaly[893]: time="2026-09-06T16:09:59.659Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4151 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:09:59.658" grpc.time_ms=0.938 pid=893 span.kind=server system=grpc gitlab # [ 279.241248] gitaly[893]: time="2026-09-06T16:09:59.669Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK diskcache=a332698e-cb82-4d7b-8bdd-68bb10764416 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:10:09.254" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:59.594" pid=893 ref=refs/keep-around/f684670ecafb28f8f83038a891201ff920ef7429 remote_ip="2001:db8:1::1" revision=f684670ecafb28f8f83038a891201ff920ef7429 span.kind=server system=grpc username=root gitlab # [ 279.282836] nixbot[2766]: {"ts": "2026-09-06T16:09:59.826Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-f684670ecafb-5", "argv0": "bwrap"} gitlab # [ 279.283734] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.731Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f3077871eb8eeb032d39d814","created_at":"2026-09-06T16:09:57.081Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:57.083Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2774,"job_status":"rescheduled","message":"PipelineProcessWorker JID-f3077871eb8eeb032d39d814: rescheduled"} gitlab # [ 279.285197] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.736Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f3077871eb8eeb032d39d814","created_at":"2026-09-06T16:09:57.081Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-e99b774fb3df4fa7","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:57.083Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f3077871eb8eeb032d39d814: done: 0.100656 sec","job_status":"done","queue_duration_s":2.552254,"scheduling_latency_s":2.552291,"gitaly_calls":2,"gitaly_duration_s":0.007591,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004398,"redis_read_bytes":59,"redis_write_bytes":6216,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000653,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000481,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_calls":4,"redis_queues_duration_s":0.000364,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2219,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001313,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":979,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000141,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000135,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000658,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":29,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":14,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055974,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.073063755,"duration_s":0.100656,"completed_at":"2026-09-06T16:09:59.736Z","load_balancing_strategy":"primary","db_duration_s":0.031049,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 279.333209] gitaly[893]: time="2026-09-06T16:09:59.669Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=46 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5326 command.oublock=8 command.real_time_ms=68 command.spawn_token_fork_ms=2 command.system_time_ms=26 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:10:09.254" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:09:59.594" grpc.time_ms=75.499 pid=893 ref=refs/keep-around/f684670ecafb28f8f83038a891201ff920ef7429 remote_ip="2001:db8:1::1" revision=f684670ecafb28f8f83038a891201ff920ef7429 span.kind=server system=grpc username=root gitlab # [ 279.333851] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:09:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 856 gitlab # [ 279.334439] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.737Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2fd6b541eb3af3735e389ba4","created_at":"2026-09-06T16:09:57.175Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-9c5a4d30d8ad4af0","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:57.181Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2fd6b541eb3af3735e389ba4: start","job_status":"start","queue_duration_s":2.555561,"scheduling_latency_s":2.555576} gitlab # [ 279.341731] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.765Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2fd6b541eb3af3735e389ba4","created_at":"2026-09-06T16:09:57.175Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-9c5a4d30d8ad4af0","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:57.181Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2fd6b541eb3af3735e389ba4: done: 0.028436 sec","job_status":"done","queue_duration_s":2.555561,"scheduling_latency_s":2.555576,"redis_calls":19,"redis_duration_s":0.003009,"redis_read_bytes":21,"redis_write_bytes":4288,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000311,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2259,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001782,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004349156,"duration_s":0.028436,"completed_at":"2026-09-06T16:09:59.765Z","load_balancing_strategy":"primary","db_duration_s":0.004685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.385350] gitaly[893]: time="2026-09-06T16:09:59.672Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:29.672" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:59.672" grpc.time_ms=0.275 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.393255] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.766Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","30f075a1224d747af0c2571c"],"class":"WebHooks::LogExecutionWorker","jid":"d4c58a2fada5b4429e7379be","created_at":"2026-09-06T16:09:58.206Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-cdd3d4a30a0140ed","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:93e4da483bc3c3a75278950084db9ad04de14492ae09708832c4736b81c5962f","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:58.213Z","job_size_bytes":2592,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-d4c58a2fada5b4429e7379be: start","job_status":"start","queue_duration_s":1.553599,"scheduling_latency_s":1.553612} gitlab # [ 279.408200] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.802Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","30f075a1224d747af0c2571c"],"class":"WebHooks::LogExecutionWorker","jid":"d4c58a2fada5b4429e7379be","created_at":"2026-09-06T16:09:58.206Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-cdd3d4a30a0140ed","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:93e4da483bc3c3a75278950084db9ad04de14492ae09708832c4736b81c5962f","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:58.213Z","job_size_bytes":2592,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-d4c58a2fada5b4429e7379be: done: 0.035847 sec","job_status":"done","queue_duration_s":1.553599,"scheduling_latency_s":1.553612,"redis_calls":10,"redis_duration_s":0.004136,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000649,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227403,"duration_s":0.035847,"completed_at":"2026-09-06T16:09:59.802Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.471728] gitaly[893]: time="2026-09-06T16:09:59.700Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:10:29.696" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:59.696" grpc.time_ms=3.109 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.472574] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.804Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","be9fc1402c340121769713b7"],"class":"WebHooks::LogExecutionWorker","jid":"376e5f9d22d7f0b1b3db9602","created_at":"2026-09-06T16:09:58.908Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bece431800c24721","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:48c901b7ec3300b5a329d86bf289f4822a44833e455783f20cb6469ddf18a460","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:58.911Z","job_size_bytes":2592,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-376e5f9d22d7f0b1b3db9602: start","job_status":"start","queue_duration_s":0.892559,"scheduling_latency_s":0.892576} gitlab # [ 279.474560] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.837Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","be9fc1402c340121769713b7"],"class":"WebHooks::LogExecutionWorker","jid":"376e5f9d22d7f0b1b3db9602","created_at":"2026-09-06T16:09:58.908Z","trace_propagation_headers":{"sentry-trace":"186322ebf63441dcbfbfbee6a5564abf-bece431800c24721","baggage":"sentry-trace_id=186322ebf63441dcbfbfbee6a5564abf,sentry-release=Unknown"},"correlation_id":"01M1VQNTRZ0AACKBRYAJ2PZ193","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:48c901b7ec3300b5a329d86bf289f4822a44833e455783f20cb6469ddf18a460","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:58.911Z","job_size_bytes":2592,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-376e5f9d22d7f0b1b3db9602: done: 0.033053 sec","job_status":"done","queue_duration_s":0.892559,"scheduling_latency_s":0.892576,"redis_calls":7,"redis_duration_s":0.004573,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004267,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033053,"completed_at":"2026-09-06T16:09:59.837Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.519567] gitaly[893]: time="2026-09-06T16:09:59.728Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:29.725" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:59.726" grpc.time_ms=1.011 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.524024] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.838Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a9983d25661b58e638125c2f","created_at":"2026-09-06T16:09:59.673Z","trace_propagation_headers":{"sentry-trace":"31ca8c1e8e734d2ebb7813376da92928-ee45f8daeb254117","baggage":"sentry-trace_id=31ca8c1e8e734d2ebb7813376da92928,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQN9Z1R5AJH1ESTW306QK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.676Z","job_size_bytes":65,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a9983d25661b58e638125c2f: start","job_status":"start","queue_duration_s":0.161295,"scheduling_latency_s":0.161313} gitlab # [ 279.575745] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.862Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a9983d25661b58e638125c2f","created_at":"2026-09-06T16:09:59.673Z","trace_propagation_headers":{"sentry-trace":"31ca8c1e8e734d2ebb7813376da92928-ee45f8daeb254117","baggage":"sentry-trace_id=31ca8c1e8e734d2ebb7813376da92928,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQN9Z1R5AJH1ESTW306QK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.676Z","job_size_bytes":65,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a9983d25661b58e638125c2f: done: 0.024561 sec","job_status":"done","queue_duration_s":0.161295,"scheduling_latency_s":0.161313,"redis_calls":7,"redis_duration_s":0.000925,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024561,"completed_at":"2026-09-06T16:09:59.862Z","load_balancing_strategy":"primary","db_duration_s":0.008799,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 279.576976] gitaly[893]: time="2026-09-06T16:09:59.872Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=17 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQN0KSFV3FDWW57CVR7WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:09.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:09:59.849" grpc.time_ms=23.342 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.588185] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.863Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"22c401e78db0fe6037502a05","created_at":"2026-09-06T16:09:59.677Z","trace_propagation_headers":{"sentry-trace":"31ca8c1e8e734d2ebb7813376da92928-ee45f8daeb254117","baggage":"sentry-trace_id=31ca8c1e8e734d2ebb7813376da92928,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQN9Z1R5AJH1ESTW306QK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.678Z","job_size_bytes":65,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-22c401e78db0fe6037502a05: start","job_status":"start","queue_duration_s":0.185202,"scheduling_latency_s":0.185217} gitlab # [ 279.603680] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.905Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"22c401e78db0fe6037502a05","created_at":"2026-09-06T16:09:59.677Z","trace_propagation_headers":{"sentry-trace":"31ca8c1e8e734d2ebb7813376da92928-ee45f8daeb254117","baggage":"sentry-trace_id=31ca8c1e8e734d2ebb7813376da92928,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQN9Z1R5AJH1ESTW306QK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.678Z","job_size_bytes":65,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-22c401e78db0fe6037502a05: done: 0.04145 sec","job_status":"done","queue_duration_s":0.185202,"scheduling_latency_s":0.185217,"redis_calls":10,"redis_duration_s":0.004033,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00337,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04145,"completed_at":"2026-09-06T16:09:59.905Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429' gitlab # [ 279.650278] gitaly[893]: time="2026-09-06T16:09:59.982Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:10:29.081" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:09:59.981" grpc.time_ms=0.713 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.653886] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.906Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"64950c2ef7ef4ac09a7b6c5c","created_at":"2026-09-06T16:09:59.685Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-c8eebbb014914f3e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.687Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-64950c2ef7ef4ac09a7b6c5c: start","job_status":"start","queue_duration_s":0.218807,"scheduling_latency_s":0.218822} gitlab # [ 279.668607] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.929Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"64950c2ef7ef4ac09a7b6c5c","created_at":"2026-09-06T16:09:59.685Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-c8eebbb014914f3e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.687Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-64950c2ef7ef4ac09a7b6c5c: done: 0.023326 sec","job_status":"done","queue_duration_s":0.218807,"scheduling_latency_s":0.218822,"redis_calls":7,"redis_duration_s":0.001387,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023326,"completed_at":"2026-09-06T16:09:59.929Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.721398] gitaly[893]: time="2026-09-06T16:10:00.005Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQN9Z1R5AJH1ESTW306QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:30.002" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:00.002" grpc.time_ms=3.27 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.721931] gitaly[893]: time="2026-09-06T16:10:00.077Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:00.077" grpc.time_ms=0.317 pid=893 span.kind=server system=grpc gitlab # [ 279.723465] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:09:59.930Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"05122bbb760008d3d0444490","created_at":"2026-09-06T16:09:59.710Z","trace_propagation_headers":{"sentry-trace":"31ca8c1e8e734d2ebb7813376da92928-ee45f8daeb254117","baggage":"sentry-trace_id=31ca8c1e8e734d2ebb7813376da92928,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQN9Z1R5AJH1ESTW306QK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.712Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-05122bbb760008d3d0444490: start","job_status":"start","queue_duration_s":0.217944,"scheduling_latency_s":0.21797} gitlab # [ 279.763762] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.012Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"05122bbb760008d3d0444490","created_at":"2026-09-06T16:09:59.710Z","trace_propagation_headers":{"sentry-trace":"31ca8c1e8e734d2ebb7813376da92928-ee45f8daeb254117","baggage":"sentry-trace_id=31ca8c1e8e734d2ebb7813376da92928,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQN9Z1R5AJH1ESTW306QK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.712Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-05122bbb760008d3d0444490: done: 0.081735 sec","job_status":"done","queue_duration_s":0.217944,"scheduling_latency_s":0.21797,"gitaly_calls":2,"gitaly_duration_s":0.00679,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.00423,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000671,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000641,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000184,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000145,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000279,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000665,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057394,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.060081099,"duration_s":0.081735,"completed_at":"2026-09-06T16:10:00.012Z","load_balancing_strategy":"primary","db_duration_s":0.016527,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 279.814214] gitaly[893]: time="2026-09-06T16:10:00.096Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:00.095" grpc.time_ms=0.335 pid=893 span.kind=server system=grpc gitlab # [ 279.814827] gitaly[893]: time="2026-09-06T16:10:00.115Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:00.114" grpc.time_ms=0.203 pid=893 span.kind=server system=grpc gitlab # [ 279.815465] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.013Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8aa5d2131591c59c8a38b07f","created_at":"2026-09-06T16:09:59.732Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-c8eebbb014914f3e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.733Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8aa5d2131591c59c8a38b07f: start","job_status":"start","queue_duration_s":0.280186,"scheduling_latency_s":0.280201} gitlab # [ 279.816060] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.185Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8aa5d2131591c59c8a38b07f","created_at":"2026-09-06T16:09:59.732Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-c8eebbb014914f3e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.733Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2774,"job_status":"rescheduled","message":"PipelineProcessWorker JID-8aa5d2131591c59c8a38b07f: rescheduled"} gitlab # [ 279.825445] gitaly[893]: time="2026-09-06T16:10:00.123Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y diskcache=ef962b7c-3ae0-48ad-9a51-3b47c53b5434 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:30.056" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:00.056" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.864959] gitaly[893]: time="2026-09-06T16:10:00.123Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=69 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7323 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=0 command.system_time_ms=42 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:30.056" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:00.056" grpc.time_ms=66.71 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.865788] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.190Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8aa5d2131591c59c8a38b07f","created_at":"2026-09-06T16:09:59.732Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-c8eebbb014914f3e","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.733Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8aa5d2131591c59c8a38b07f: done: 0.176649 sec","job_status":"done","queue_duration_s":0.280186,"scheduling_latency_s":0.280201,"gitaly_calls":3,"gitaly_duration_s":0.072637,"redis_calls":75,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008069999999999999,"redis_read_bytes":159,"redis_write_bytes":21255,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000558,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000603,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":25,"redis_queues_duration_s":0.002067,"redis_queues_read_bytes":30,"redis_queues_write_bytes":14465,"redis_queues_metadata_calls":20,"redis_queues_metadata_duration_s":0.002681,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3301,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000236,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000314,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081771,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.155382597,"duration_s":0.176649,"completed_at":"2026-09-06T16:10:00.190Z","load_balancing_strategy":"primary","db_duration_s":0.021485,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 279.890454] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.191Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"9f446af6f2e40bfd8fd9366c","created_at":"2026-09-06T16:09:59.757Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-a507d348b8f04afb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.760Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-9f446af6f2e40bfd8fd9366c: start","job_status":"start","queue_duration_s":0.43017,"scheduling_latency_s":0.430185} gitlab # [ 279.905663] gitaly[893]: time="2026-09-06T16:10:00.159Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:10:30.158" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:00.158" grpc.time_ms=0.657 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.949729] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.243Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"9f446af6f2e40bfd8fd9366c","created_at":"2026-09-06T16:09:59.757Z","trace_propagation_headers":{"sentry-trace":"24567ca3a43246948e0ea3c5f8fe2e4e-a507d348b8f04afb","baggage":"sentry-trace_id=24567ca3a43246948e0ea3c5f8fe2e4e,sentry-release=Unknown"},"correlation_id":"01M1VQNNRRN45T1NSMNR97ZGDW","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.760Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-9f446af6f2e40bfd8fd9366c: done: 0.052694 sec","job_status":"done","queue_duration_s":0.43017,"scheduling_latency_s":0.430185,"redis_calls":11,"redis_duration_s":0.001411,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212597,"duration_s":0.052694,"completed_at":"2026-09-06T16:10:00.243Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.988898] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.244Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"1abaa23f65bff3f7c6ca8714","created_at":"2026-09-06T16:09:59.971Z","trace_propagation_headers":{"sentry-trace":"31ca8c1e8e734d2ebb7813376da92928-d4a6a7a757ce4733","baggage":"sentry-trace_id=31ca8c1e8e734d2ebb7813376da92928,sentry-release=Unknown"},"correlation_id":"01M1VQQN9Z1R5AJH1ESTW306QK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.972Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1abaa23f65bff3f7c6ca8714: start","job_status":"start","queue_duration_s":0.272175,"scheduling_latency_s":0.272189} gitlab # [ 279.989695] gitaly[893]: time="2026-09-06T16:10:00.180Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQNVA5VYN2CVBSCGTSXX7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:30.179" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:00.180" grpc.time_ms=0.502 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.990405] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.264Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"1abaa23f65bff3f7c6ca8714","created_at":"2026-09-06T16:09:59.971Z","trace_propagation_headers":{"sentry-trace":"31ca8c1e8e734d2ebb7813376da92928-d4a6a7a757ce4733","baggage":"sentry-trace_id=31ca8c1e8e734d2ebb7813376da92928,sentry-release=Unknown"},"correlation_id":"01M1VQQN9Z1R5AJH1ESTW306QK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:09:59.972Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1abaa23f65bff3f7c6ca8714: done: 0.019895 sec","job_status":"done","queue_duration_s":0.272175,"scheduling_latency_s":0.272189,"redis_calls":7,"redis_duration_s":0.000887,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019895,"completed_at":"2026-09-06T16:10:00.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.038703] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.265Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d6e4d2cdfdcdf3ea24287568","created_at":"2026-09-06T16:10:00.051Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.053Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d6e4d2cdfdcdf3ea24287568: start","job_status":"start","queue_duration_s":0.212208,"scheduling_latency_s":0.212222} gitlab # [ 280.039484] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.288Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d6e4d2cdfdcdf3ea24287568","created_at":"2026-09-06T16:10:00.051Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.053Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d6e4d2cdfdcdf3ea24287568: done: 0.022677 sec","job_status":"done","queue_duration_s":0.212208,"scheduling_latency_s":0.212222,"redis_calls":7,"redis_duration_s":0.00091,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022677,"completed_at":"2026-09-06T16:10:00.288Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00396,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 280.081488] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.289Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"0d90dc87d6405b02610b943a","created_at":"2026-09-06T16:10:00.127Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.128Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-0d90dc87d6405b02610b943a: start","job_status":"start","queue_duration_s":0.160147,"scheduling_latency_s":0.160162} gitlab # [ 280.083644] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.304Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"0d90dc87d6405b02610b943a","created_at":"2026-09-06T16:10:00.127Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.128Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-0d90dc87d6405b02610b943a: done: 0.015164 sec","job_status":"done","queue_duration_s":0.160147,"scheduling_latency_s":0.160162,"redis_calls":7,"redis_duration_s":0.0009220000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015164,"completed_at":"2026-09-06T16:10:00.304Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.132691] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.305Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ca267cd13bc211d567bdb811","created_at":"2026-09-06T16:10:00.132Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.133Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ca267cd13bc211d567bdb811: start","job_status":"start","queue_duration_s":0.172017,"scheduling_latency_s":0.172031} gitlab # [ 280.134943] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.324Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ca267cd13bc211d567bdb811","created_at":"2026-09-06T16:10:00.132Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.133Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ca267cd13bc211d567bdb811: done: 0.019134 sec","job_status":"done","queue_duration_s":0.172017,"scheduling_latency_s":0.172031,"redis_calls":7,"redis_duration_s":0.002885,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019134,"completed_at":"2026-09-06T16:10:00.324Z","load_balancing_strategy":"primary","db_duration_s":0.002527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.182639] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.325Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"34cb76d6132fc1db2892620e","created_at":"2026-09-06T16:10:00.137Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.138Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-34cb76d6132fc1db2892620e: start","job_status":"start","queue_duration_s":0.187165,"scheduling_latency_s":0.187186} gitlab # [ 280.185072] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.345Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"34cb76d6132fc1db2892620e","created_at":"2026-09-06T16:10:00.137Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.138Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-34cb76d6132fc1db2892620e: done: 0.020436 sec","job_status":"done","queue_duration_s":0.187165,"scheduling_latency_s":0.187186,"redis_calls":10,"redis_duration_s":0.001335,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000343,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000317,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020436,"completed_at":"2026-09-06T16:10:00.345Z","load_balancing_strategy":"primary","db_duration_s":0.002824,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.231120] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.346Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2030e72175fff245ccd0735c","created_at":"2026-09-06T16:10:00.138Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.139Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2030e72175fff245ccd0735c: start","job_status":"start","queue_duration_s":0.207367,"scheduling_latency_s":0.207382} gitlab # [ 280.269895] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.421Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2030e72175fff245ccd0735c","created_at":"2026-09-06T16:10:00.138Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.139Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2030e72175fff245ccd0735c: done: 0.074684 sec","job_status":"done","queue_duration_s":0.207367,"scheduling_latency_s":0.207382,"redis_calls":7,"redis_duration_s":0.001025,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.074684,"completed_at":"2026-09-06T16:10:00.421Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002915,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.274530] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.422Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"dd994569735c4d8629868407","created_at":"2026-09-06T16:10:00.139Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.141Z","job_size_bytes":19,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-dd994569735c4d8629868407: start","job_status":"start","queue_duration_s":0.281432,"scheduling_latency_s":0.28145} gitlab # [ 280.288915] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.437Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"dd994569735c4d8629868407","created_at":"2026-09-06T16:10:00.139Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.141Z","job_size_bytes":19,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-dd994569735c4d8629868407: done: 0.014904 sec","job_status":"done","queue_duration_s":0.281432,"scheduling_latency_s":0.28145,"redis_calls":8,"redis_duration_s":0.001011,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000227,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014904,"completed_at":"2026-09-06T16:10:00.437Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.352750] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.438Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"43778ee3739a32251ea56ee9","created_at":"2026-09-06T16:10:00.141Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.142Z","job_size_bytes":66,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-43778ee3739a32251ea56ee9: start","job_status":"start","queue_duration_s":0.295751,"scheduling_latency_s":0.295765} gitlab # [ 280.353531] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.451Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"43778ee3739a32251ea56ee9","created_at":"2026-09-06T16:10:00.141Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.142Z","job_size_bytes":66,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-43778ee3739a32251ea56ee9: done: 0.012917 sec","job_status":"done","queue_duration_s":0.295751,"scheduling_latency_s":0.295765,"redis_calls":7,"redis_duration_s":0.000878,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012917,"completed_at":"2026-09-06T16:10:00.451Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.379947] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.452Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"6592ae9a5d1df49c2629f658","created_at":"2026-09-06T16:10:00.143Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.143Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6592ae9a5d1df49c2629f658: start","job_status":"start","queue_duration_s":0.308522,"scheduling_latency_s":0.308534} gitlab # [ 280.394187] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.472Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"6592ae9a5d1df49c2629f658","created_at":"2026-09-06T16:10:00.143Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.143Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6592ae9a5d1df49c2629f658: done: 0.019573 sec","job_status":"done","queue_duration_s":0.308522,"scheduling_latency_s":0.308534,"redis_calls":7,"redis_duration_s":0.000877,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000227,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019573,"completed_at":"2026-09-06T16:10:00.472Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.451959] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.472Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9ed36631454059d4a85f336e","created_at":"2026-09-06T16:10:00.147Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.148Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9ed36631454059d4a85f336e: start","job_status":"start","queue_duration_s":0.32391,"scheduling_latency_s":0.323924} gitlab # [ 280.452648] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.488Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9ed36631454059d4a85f336e","created_at":"2026-09-06T16:10:00.147Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.148Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9ed36631454059d4a85f336e: done: 0.015645 sec","job_status":"done","queue_duration_s":0.32391,"scheduling_latency_s":0.323924,"redis_calls":7,"redis_duration_s":0.000896,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015645,"completed_at":"2026-09-06T16:10:00.488Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003415,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.483869] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"956f1712adba1fe4806a1d13","created_at":"2026-09-06T16:10:00.149Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.150Z","job_size_bytes":6,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-956f1712adba1fe4806a1d13: start","job_status":"start","queue_duration_s":0.339183,"scheduling_latency_s":0.339206} gitlab # [ 280.525486] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.497Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"956f1712adba1fe4806a1d13","created_at":"2026-09-06T16:10:00.149Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.150Z","job_size_bytes":6,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-956f1712adba1fe4806a1d13: done: 0.0086 sec","job_status":"done","queue_duration_s":0.339183,"scheduling_latency_s":0.339206,"redis_calls":7,"redis_duration_s":0.000846,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0086,"completed_at":"2026-09-06T16:10:00.497Z","load_balancing_strategy":"primary","db_duration_s":0.000292,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.566289] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.498Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1dbc8483b241be63a971deed","created_at":"2026-09-06T16:10:00.150Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.151Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1dbc8483b241be63a971deed: start","job_status":"start","queue_duration_s":0.347465,"scheduling_latency_s":0.347477} gitlab # [ 280.567071] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.512Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1dbc8483b241be63a971deed","created_at":"2026-09-06T16:10:00.150Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.151Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1dbc8483b241be63a971deed: done: 0.01381 sec","job_status":"done","queue_duration_s":0.347465,"scheduling_latency_s":0.347477,"redis_calls":7,"redis_duration_s":0.000858,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000229,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01381,"completed_at":"2026-09-06T16:10:00.512Z","load_balancing_strategy":"primary","db_duration_s":0.001932,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.611498] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.513Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ce5d8b178fa87c822a9182b0","created_at":"2026-09-06T16:10:00.185Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.187Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ce5d8b178fa87c822a9182b0: start","job_status":"start","queue_duration_s":0.326373,"scheduling_latency_s":0.326386} gitlab # [ 280.612222] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:00.531Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ce5d8b178fa87c822a9182b0","created_at":"2026-09-06T16:10:00.185Z","trace_propagation_headers":{"sentry-trace":"e3291443afcc4cd588a6e4cb967c69fd-acfcfa13154844b3","baggage":"sentry-trace_id=e3291443afcc4cd588a6e4cb967c69fd,sentry-release=Unknown"},"correlation_id":"01M1VQNVA5VYN2CVBSCGTSXX7Y","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:00.187Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ce5d8b178fa87c822a9182b0: done: 0.018284 sec","job_status":"done","queue_duration_s":0.326373,"scheduling_latency_s":0.326386,"redis_calls":8,"redis_duration_s":0.0015400000000000001,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000337,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018284,"completed_at":"2026-09-06T16:10:00.531Z","load_balancing_strategy":"primary","db_duration_s":0.004085,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 280.681714] nixbot[2766]: {"ts": "2026-09-06T16:10:01.281Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f12cff88c90489b6aedd8947843349145d58ca39 \"HTTP/1.1 201 Created\""} gitlab # [ 280.688335] nixbot[2766]: INFO: ::ffff:127.0.0.1:40304 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f684670ecafb28f8f83038a891201ff920ef7429', in 1.02 seconds) (finished: subtest: push delivers the webhook, in 62.13 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses gitlab # [ 280.728350] nix-daemon[2996]: accepted connection from pid 3633, user nixbot gitlab # [ 280.790536] gitaly[893]: time="2026-09-06T16:10:01.390Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQQ7ZDT2G1DEZK87CYMVM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:31.389" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:01.389" grpc.time_ms=0.672 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.825448] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 81 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses, in 0.12 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses gitlab # [ 281.934122] gitaly[893]: time="2026-09-06T16:10:02.534Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQRB34AXE2S4D994RTVCQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:32.532" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:02.532" grpc.time_ms=0.8 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.005379] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 137 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses, in 0.18 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses gitlab # [ 283.103419] gitaly[893]: time="2026-09-06T16:10:03.703Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQSG0WDAHTMBX8VYZKH28 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:33.701" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:03.702" grpc.time_ms=0.906 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.147701] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 99 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses, in 0.14 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses gitlab # [ 284.237600] gitaly[893]: time="2026-09-06T16:10:04.837Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQTKJAT2V5X22S6THDN68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:34.835" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:04.836" grpc.time_ms=0.845 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.266487] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 79 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses, in 0.12 seconds) gitlab # [ 285.092921] nix-daemon[2996]: accepted connection from pid 3668, user nixbot gitlab # [ 285.106117] nix-daemon[2996]: accepted connection from pid 3633, user nixbot gitlab # [ 285.196250] gitaly[893]: time="2026-09-06T16:10:05.796Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:35.795" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:05.795" grpc.time_ms=0.842 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.248688] gitaly[893]: time="2026-09-06T16:10:05.848Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:15.047" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:05.837" grpc.time_ms=11.245 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses gitlab # [ 285.275875] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:05.875Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6dd0d29d6095766f701c5ea5","created_at":"2026-09-06T16:10:05.871Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-c1c58b0b19dd4691","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:05.874Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6dd0d29d6095766f701c5ea5: start","job_status":"start","queue_duration_s":0.001384,"scheduling_latency_s":0.001415} gitlab # [ 285.298727] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 158 gitlab # [ 285.299850] nixbot[2766]: {"ts": "2026-09-06T16:10:05.899Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f684670ecafb28f8f83038a891201ff920ef7429 \"HTTP/1.1 201 Created\""} gitlab # [ 285.351375] gitaly[893]: time="2026-09-06T16:10:05.951Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVPJDG62W13CTDJPN4B6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:35.350" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:05.950" grpc.time_ms=0.953 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.368626] gitaly[893]: time="2026-09-06T16:10:05.969Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVQ2ZP10T89G204KNP8P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:35.068" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:05.968" grpc.time_ms=0.361 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.383102] gitaly[893]: time="2026-09-06T16:10:05.983Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVPJDG62W13CTDJPN4B6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:15.137" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:05.977" grpc.time_ms=5.727 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.397455] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 75 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses, in 0.13 seconds) gitlab # [ 285.426729] gitaly[893]: time="2026-09-06T16:10:06.027Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:06.027" grpc.time_ms=0.294 pid=893 span.kind=server system=grpc gitlab # [ 285.428058] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:06 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 122 gitlab # [ 285.428992] nixbot[2766]: {"ts": "2026-09-06T16:10:06.029Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f684670ecafb28f8f83038a891201ff920ef7429 \"HTTP/1.1 201 Created\""} gitlab # [ 285.459785] gitaly[893]: time="2026-09-06T16:10:06.060Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:06.060" grpc.time_ms=0.241 pid=893 span.kind=server system=grpc gitlab # [ 285.476745] gitaly[893]: time="2026-09-06T16:10:06.077Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVTVPKJAVG5T0FZC6JEA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:36.076" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:06.076" grpc.time_ms=0.383 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.493476] gitaly[893]: time="2026-09-06T16:10:06.094Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:06.094" grpc.time_ms=0.203 pid=893 span.kind=server system=grpc gitlab # [ 285.502191] gitaly[893]: time="2026-09-06T16:10:06.102Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVTVPKJAVG5T0FZC6JEA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:10:16.097" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.097" grpc.time_ms=5.211 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.515832] gitaly[893]: time="2026-09-06T16:10:06.116Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:36.009" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.009" operations="[repack_refs]" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.523662] gitaly[893]: time="2026-09-06T16:10:06.124Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC diskcache=7d76d2d7-8734-43bb-969e-3f9709d743f4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:36.009" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.009" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.525386] gitaly[893]: time="2026-09-06T16:10:06.124Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=81 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6889 command.oublock=0 command.real_time_ms=106 command.spawn_token_fork_ms=0 command.system_time_ms=56 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:36.009" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.009" grpc.time_ms=115.185 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.533581] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:06 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f684670ecafb28f8f83038a891201ff920ef7429 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 91 gitlab # [ 285.534464] nixbot[2766]: {"ts": "2026-09-06T16:10:06.135Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f684670ecafb28f8f83038a891201ff920ef7429 \"HTTP/1.1 201 Created\""} gitlab # [ 285.552354] gitaly[893]: time="2026-09-06T16:10:06.153Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:36.009" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.009" pid=893 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-06 16:08:52.68074725 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 285.563914] gitaly[893]: time="2026-09-06T16:10:06.164Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:36.009" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.009" pid=893 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 285.575690] gitaly[893]: time="2026-09-06T16:10:06.164Z" level=info msg="optimized repository" clean_stale_data_time=11.257017ms clean_work_trees_time="65.092µs" commit_graph_time="3.352µs" correlation_id=01M1VQQVHBPX97S46FMHDSQHBC gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="56.432µs" pid=893 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="3.632µs" storage_name=default system=housekeeping total_time=11.47576ms gitlab # [ 285.581146] gitaly[893]: time="2026-09-06T16:10:06.164Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:36.009" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.009" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.614899] gitaly[893]: time="2026-09-06T16:10:06.215Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:10:36.214" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:06.214" grpc.time_ms=0.788 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.634667] gitaly[893]: time="2026-09-06T16:10:06.235Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:36.234" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:06.234" grpc.time_ms=0.469 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.651907] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.252Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"263d11e16193662572190abe","created_at":"2026-09-06T16:10:06.251Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"6dd0d29d6095766f701c5ea5","job_size_bytes":3,"pid":2774,"job_status":"deduplicated","message":"PipelineProcessWorker JID-263d11e16193662572190abe: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 285.652967] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.253Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6dd0d29d6095766f701c5ea5","created_at":"2026-09-06T16:10:05.871Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-c1c58b0b19dd4691","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:05.874Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2774,"job_status":"rescheduled","message":"PipelineProcessWorker JID-6dd0d29d6095766f701c5ea5: rescheduled"} gitlab # [ 285.657734] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.258Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6dd0d29d6095766f701c5ea5","created_at":"2026-09-06T16:10:05.871Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-c1c58b0b19dd4691","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:05.874Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6dd0d29d6095766f701c5ea5: done: 0.382746 sec","job_status":"done","queue_duration_s":0.001384,"scheduling_latency_s":0.001415,"gitaly_calls":3,"gitaly_duration_s":0.132112,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.054478,"redis_read_bytes":239,"redis_write_bytes":21784,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001798,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00088,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.013839,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14469,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.032406,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000462,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00047,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002825,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.025,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.027,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.125386,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.315610224,"duration_s":0.382746,"completed_at":"2026-09-06T16:10:06.258Z","load_balancing_strategy":"primary","db_duration_s":0.047232,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 285.711068] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.260Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"e5de370b0cc5695f3f38547f","created_at":"2026-09-06T16:10:06.003Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.005Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-e5de370b0cc5695f3f38547f: start","job_status":"start","queue_duration_s":0.254623,"scheduling_latency_s":0.254639} gitlab # [ 285.711841] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.289Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"e5de370b0cc5695f3f38547f","created_at":"2026-09-06T16:10:06.003Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.005Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-e5de370b0cc5695f3f38547f: done: 0.028997 sec","job_status":"done","queue_duration_s":0.254623,"scheduling_latency_s":0.254639,"redis_calls":7,"redis_duration_s":0.0036119999999999998,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002345,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028997,"completed_at":"2026-09-06T16:10:06.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004876,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 285.755748] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.290Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6f152dfafa7256354de5faac","created_at":"2026-09-06T16:10:06.130Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.131Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6f152dfafa7256354de5faac: start","job_status":"start","queue_duration_s":0.158089,"scheduling_latency_s":0.158105} gitlab # [ 285.756673] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.306Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6f152dfafa7256354de5faac","created_at":"2026-09-06T16:10:06.130Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.131Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6f152dfafa7256354de5faac: done: 0.016721 sec","job_status":"done","queue_duration_s":0.158089,"scheduling_latency_s":0.158105,"redis_calls":7,"redis_duration_s":0.000902,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016721,"completed_at":"2026-09-06T16:10:06.306Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002922,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.804893] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.308Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"73ebdc17a19811c14f46b820","created_at":"2026-09-06T16:10:06.138Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.141Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-73ebdc17a19811c14f46b820: start","job_status":"start","queue_duration_s":0.166879,"scheduling_latency_s":0.166894} gitlab # [ 285.805705] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.324Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"73ebdc17a19811c14f46b820","created_at":"2026-09-06T16:10:06.138Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.141Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-73ebdc17a19811c14f46b820: done: 0.016666 sec","job_status":"done","queue_duration_s":0.166879,"scheduling_latency_s":0.166894,"redis_calls":7,"redis_duration_s":0.000975,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016666,"completed_at":"2026-09-06T16:10:06.324Z","load_balancing_strategy":"primary","db_duration_s":0.002582,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.848753] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.325Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"978a4b1c032212f189545aa2","created_at":"2026-09-06T16:10:06.149Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.150Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-978a4b1c032212f189545aa2: start","job_status":"start","queue_duration_s":0.174687,"scheduling_latency_s":0.174701} gitlab # [ 285.851867] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.348Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"978a4b1c032212f189545aa2","created_at":"2026-09-06T16:10:06.149Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.150Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-978a4b1c032212f189545aa2: done: 0.022773 sec","job_status":"done","queue_duration_s":0.174687,"scheduling_latency_s":0.174701,"redis_calls":10,"redis_duration_s":0.0020369999999999997,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000667,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022773,"completed_at":"2026-09-06T16:10:06.348Z","load_balancing_strategy":"primary","db_duration_s":0.005969,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.893819] gitaly[893]: time="2026-09-06T16:10:06.427Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQW5HNAAVPEWAGRJEQMJV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:10:06.421" grpc.time_ms=5.928 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 285.902036] gitaly[893]: time="2026-09-06T16:10:06.468Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQW6XR2AQYXFW1ZSXGN18 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:10:06.462" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 285.910738] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 42 gitlab # [ 285.912435] gitlab-workhorse[827]: time="2026-09-06T16:10:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 285.915028] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 39 gitlab # [ 285.916817] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.350Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"06ff6c65f43328a404985542","created_at":"2026-09-06T16:10:06.151Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.153Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-06ff6c65f43328a404985542: start","job_status":"start","queue_duration_s":0.196917,"scheduling_latency_s":0.196947} gitlab # [ 285.960177] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.375Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"06ff6c65f43328a404985542","created_at":"2026-09-06T16:10:06.151Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.153Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-06ff6c65f43328a404985542: done: 0.025117 sec","job_status":"done","queue_duration_s":0.196917,"scheduling_latency_s":0.196947,"redis_calls":7,"redis_duration_s":0.0015069999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025117,"completed_at":"2026-09-06T16:10:06.375Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.968210] gitaly[893]: time="2026-09-06T16:10:06.468Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQW6XR2AQYXFW1ZSXGN18 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:10:06.462" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 286.006483] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 40 gitlab # [ 286.007179] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.376Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"42abdf71ca63d16ec88198a2","created_at":"2026-09-06T16:10:06.164Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.192Z","job_size_bytes":19,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-42abdf71ca63d16ec88198a2: start","job_status":"start","queue_duration_s":0.183614,"scheduling_latency_s":0.183629} gitlab # [ 286.007933] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.396Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"42abdf71ca63d16ec88198a2","created_at":"2026-09-06T16:10:06.164Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.192Z","job_size_bytes":19,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-42abdf71ca63d16ec88198a2: done: 0.020238 sec","job_status":"done","queue_duration_s":0.183614,"scheduling_latency_s":0.183629,"redis_calls":8,"redis_duration_s":0.004434,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004156,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000278,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020238,"completed_at":"2026-09-06T16:10:06.396Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.063770] gitaly[893]: time="2026-09-06T16:10:06.468Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQW6XR2AQYXFW1ZSXGN18 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:10:06.462" grpc.time_ms=6.3 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 286.064615] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.397Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7c0fff35620c387ffae82887","created_at":"2026-09-06T16:10:06.193Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.195Z","job_size_bytes":66,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7c0fff35620c387ffae82887: start","job_status":"start","queue_duration_s":0.201756,"scheduling_latency_s":0.201771} gitlab # [ 286.065183] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.414Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7c0fff35620c387ffae82887","created_at":"2026-09-06T16:10:06.193Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.195Z","job_size_bytes":66,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7c0fff35620c387ffae82887: done: 0.017064 sec","job_status":"done","queue_duration_s":0.201756,"scheduling_latency_s":0.201771,"redis_calls":7,"redis_duration_s":0.001325,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017064,"completed_at":"2026-09-06T16:10:06.414Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.098742] gitaly[893]: time="2026-09-06T16:10:06.530Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQW8T3VWEX6Q4RJ4GY1CE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:10:06.524" grpc.time_ms=6.22 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 286.110641] gitaly[893]: time="2026-09-06T16:10:06.589Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:06.588" grpc.time_ms=0.291 pid=893 span.kind=server system=grpc gitlab # [ 286.148747] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.415Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6546be1f4882f1fc7ce7520f","created_at":"2026-09-06T16:10:06.196Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.198Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6546be1f4882f1fc7ce7520f: start","job_status":"start","queue_duration_s":0.217074,"scheduling_latency_s":0.217089} gitlab # [ 286.149541] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.435Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6546be1f4882f1fc7ce7520f","created_at":"2026-09-06T16:10:06.196Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.198Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6546be1f4882f1fc7ce7520f: done: 0.020412 sec","job_status":"done","queue_duration_s":0.217074,"scheduling_latency_s":0.217089,"redis_calls":7,"redis_duration_s":0.0008990000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020412,"completed_at":"2026-09-06T16:10:06.435Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.197824] gitaly[893]: time="2026-09-06T16:10:06.606Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:06.605" grpc.time_ms=0.296 pid=893 span.kind=server system=grpc gitlab # [ 286.198694] gitaly[893]: time="2026-09-06T16:10:06.627Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:10:06.627" grpc.time_ms=0.254 pid=893 span.kind=server system=grpc gitlab # [ 286.199200] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.436Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"c1a914fe1cb84c584ec69438","created_at":"2026-09-06T16:10:06.202Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.203Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c1a914fe1cb84c584ec69438: start","job_status":"start","queue_duration_s":0.233228,"scheduling_latency_s":0.233242} gitlab # [ 286.200601] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.457Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"c1a914fe1cb84c584ec69438","created_at":"2026-09-06T16:10:06.202Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.203Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c1a914fe1cb84c584ec69438: done: 0.020812 sec","job_status":"done","queue_duration_s":0.233228,"scheduling_latency_s":0.233242,"redis_calls":7,"redis_duration_s":0.0017659999999999998,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001499,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020812,"completed_at":"2026-09-06T16:10:06.457Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.246831] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.458Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0a560311b0526eeb05adffb3","created_at":"2026-09-06T16:10:06.204Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.204Z","job_size_bytes":6,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0a560311b0526eeb05adffb3: start","job_status":"start","queue_duration_s":0.253695,"scheduling_latency_s":0.253709} gitlab # [ 286.251725] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.467Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0a560311b0526eeb05adffb3","created_at":"2026-09-06T16:10:06.204Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.204Z","job_size_bytes":6,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0a560311b0526eeb05adffb3: done: 0.00901 sec","job_status":"done","queue_duration_s":0.253695,"scheduling_latency_s":0.253709,"redis_calls":7,"redis_duration_s":0.001083,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.00901,"completed_at":"2026-09-06T16:10:06.467Z","load_balancing_strategy":"primary","db_duration_s":0.000247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.307136] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.468Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"cb0efbfc3eefb69b4b6dfa7e","created_at":"2026-09-06T16:10:06.205Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.206Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-cb0efbfc3eefb69b4b6dfa7e: start","job_status":"start","queue_duration_s":0.261737,"scheduling_latency_s":0.261751} gitlab # [ 286.307906] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"cb0efbfc3eefb69b4b6dfa7e","created_at":"2026-09-06T16:10:06.205Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.206Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-cb0efbfc3eefb69b4b6dfa7e: done: 0.021093 sec","job_status":"done","queue_duration_s":0.261737,"scheduling_latency_s":0.261751,"redis_calls":7,"redis_duration_s":0.001148,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021093,"completed_at":"2026-09-06T16:10:06.489Z","load_balancing_strategy":"primary","db_duration_s":0.005403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.348532] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:06.491Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"1250a4740ec59f2f5162801f","created_at":"2026-09-06T16:10:06.253Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.255Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1250a4740ec59f2f5162801f: start","job_status":"start","queue_duration_s":0.236404,"scheduling_latency_s":0.236422} gitlab # [ 286.393163] gitaly[893]: time="2026-09-06T16:10:06.993Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQWA48YPFGVG3ZFPH41R2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:10:06.574" pid=893 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses gitlab # [ 286.402834] gitaly[893]: time="2026-09-06T16:10:06.993Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=417 command.spawn_token_fork_ms=412 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQWA48YPFGVG3ZFPH41R2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:10:06.574" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 286.403406] gitlab-workhorse[827]: time="2026-09-06T16:10:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 286.404058] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 461 gitlab # [ 286.404521] gitaly[893]: time="2026-09-06T16:10:06.993Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=417 command.spawn_token_fork_ms=412 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQWA48YPFGVG3ZFPH41R2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:10:06.574" grpc.time_ms=419.684 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 286.405125] gitaly[893]: time="2026-09-06T16:10:06.997Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC diskcache=7d117626-da98-452b-ac6d-6b6b5ac2e0b2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:36.554" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.554" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 286.405769] gitaly[893]: time="2026-09-06T16:10:06.998Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=823296 command.minflt=7216 command.oublock=0 command.real_time_ms=436 command.spawn_token_fork_ms=1 command.system_time_ms=36 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:10:36.554" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:10:06.554" grpc.time_ms=443.172 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 286.451300] gitaly[893]: time="2026-09-06T16:10:07.052Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:10:37.050" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:07.051" grpc.time_ms=0.781 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 286.470537] gitaly[893]: time="2026-09-06T16:10:07.071Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQWSYM5SAD4HFQ0AAM89M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:37.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:07.070" grpc.time_ms=0.473 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 286.480163] gitaly[893]: time="2026-09-06T16:10:07.080Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VQQVHBPX97S46FMHDSQHBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:10:37.079" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:10:07.080" grpc.time_ms=0.425 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 286.488656] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.089Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"1250a4740ec59f2f5162801f","created_at":"2026-09-06T16:10:06.253Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-2abe686eb885439f","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.255Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1250a4740ec59f2f5162801f: done: 0.597545 sec","job_status":"done","queue_duration_s":0.236404,"scheduling_latency_s":0.236422,"gitaly_calls":3,"gitaly_duration_s":0.471668,"redis_calls":65,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.012202,"redis_read_bytes":144,"redis_write_bytes":16881,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001861,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000822,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":20,"redis_queues_duration_s":0.001607,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11230,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.003057,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000216,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000341,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002437,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082168,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.567032402,"duration_s":0.597545,"completed_at":"2026-09-06T16:10:07.089Z","load_balancing_strategy":"primary","db_duration_s":0.037197,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 286.498705] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.090Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"e25309ed80686030e7f506c7","created_at":"2026-09-06T16:10:06.549Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.550Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-e25309ed80686030e7f506c7: start","job_status":"start","queue_duration_s":0.539324,"scheduling_latency_s":0.539341} gitlab # [ 286.514406] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.114Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"e25309ed80686030e7f506c7","created_at":"2026-09-06T16:10:06.549Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:06.550Z","job_size_bytes":23,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-e25309ed80686030e7f506c7: done: 0.02471 sec","job_status":"done","queue_duration_s":0.539324,"scheduling_latency_s":0.539341,"redis_calls":7,"redis_duration_s":0.0008979999999999999,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000243,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02471,"completed_at":"2026-09-06T16:10:07.114Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004428,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 286.562256] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.115Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4a3c7ce4534beb2b4ae310ae","created_at":"2026-09-06T16:10:07.003Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.005Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4a3c7ce4534beb2b4ae310ae: start","job_status":"start","queue_duration_s":0.110326,"scheduling_latency_s":0.110361} gitlab # [ 286.564800] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.131Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4a3c7ce4534beb2b4ae310ae","created_at":"2026-09-06T16:10:07.003Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.005Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4a3c7ce4534beb2b4ae310ae: done: 0.015282 sec","job_status":"done","queue_duration_s":0.110326,"scheduling_latency_s":0.110361,"redis_calls":7,"redis_duration_s":0.000885,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015282,"completed_at":"2026-09-06T16:10:07.131Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.603068] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.132Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d600e5c8b0bb6a7d45bfb170","created_at":"2026-09-06T16:10:07.010Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.012Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d600e5c8b0bb6a7d45bfb170: start","job_status":"start","queue_duration_s":0.119931,"scheduling_latency_s":0.119945} gitlab # [ 286.646721] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.147Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d600e5c8b0bb6a7d45bfb170","created_at":"2026-09-06T16:10:07.010Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.012Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d600e5c8b0bb6a7d45bfb170: done: 0.015694 sec","job_status":"done","queue_duration_s":0.119931,"scheduling_latency_s":0.119945,"redis_calls":7,"redis_duration_s":0.000879,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015694,"completed_at":"2026-09-06T16:10:07.147Z","load_balancing_strategy":"primary","db_duration_s":0.002525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.652238] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.148Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"72b1fe2d83a9ed6dd21b9d39","created_at":"2026-09-06T16:10:07.016Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.017Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-72b1fe2d83a9ed6dd21b9d39: start","job_status":"start","queue_duration_s":0.131133,"scheduling_latency_s":0.131146} gitlab # [ 286.694018] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.166Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"72b1fe2d83a9ed6dd21b9d39","created_at":"2026-09-06T16:10:07.016Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.017Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-72b1fe2d83a9ed6dd21b9d39: done: 0.018028 sec","job_status":"done","queue_duration_s":0.131133,"scheduling_latency_s":0.131146,"redis_calls":10,"redis_duration_s":0.0014559999999999998,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000309,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018028,"completed_at":"2026-09-06T16:10:07.166Z","load_balancing_strategy":"primary","db_duration_s":0.002866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.703554] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.167Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"be43bce954a92cfdfed24c33","created_at":"2026-09-06T16:10:07.017Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.018Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-be43bce954a92cfdfed24c33: start","job_status":"start","queue_duration_s":0.148798,"scheduling_latency_s":0.148812} gitlab # [ 286.746138] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.182Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"be43bce954a92cfdfed24c33","created_at":"2026-09-06T16:10:07.017Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.018Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-be43bce954a92cfdfed24c33: done: 0.015086 sec","job_status":"done","queue_duration_s":0.148798,"scheduling_latency_s":0.148812,"redis_calls":7,"redis_duration_s":0.000902,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015086,"completed_at":"2026-09-06T16:10:07.182Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002162,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.785220] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.183Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6a173f583be350808ed9996a","created_at":"2026-09-06T16:10:07.019Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.020Z","job_size_bytes":19,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6a173f583be350808ed9996a: start","job_status":"start","queue_duration_s":0.163011,"scheduling_latency_s":0.163024} gitlab # [ 286.785880] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.197Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6a173f583be350808ed9996a","created_at":"2026-09-06T16:10:07.019Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.020Z","job_size_bytes":19,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6a173f583be350808ed9996a: done: 0.014058 sec","job_status":"done","queue_duration_s":0.163011,"scheduling_latency_s":0.163024,"redis_calls":8,"redis_duration_s":0.001079,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014058,"completed_at":"2026-09-06T16:10:07.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002006,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.825277] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.198Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f9b16e6332030dffc59e18c8","created_at":"2026-09-06T16:10:07.021Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.021Z","job_size_bytes":66,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f9b16e6332030dffc59e18c8: start","job_status":"start","queue_duration_s":0.176558,"scheduling_latency_s":0.176572} gitlab # [ 286.825425] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.211Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f9b16e6332030dffc59e18c8","created_at":"2026-09-06T16:10:07.021Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.021Z","job_size_bytes":66,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f9b16e6332030dffc59e18c8: done: 0.012664 sec","job_status":"done","queue_duration_s":0.176558,"scheduling_latency_s":0.176572,"redis_calls":7,"redis_duration_s":0.00086,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000233,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012664,"completed_at":"2026-09-06T16:10:07.211Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.886774] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.212Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"1c0dd5f669c9a9da9f87e056","created_at":"2026-09-06T16:10:07.022Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.023Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1c0dd5f669c9a9da9f87e056: start","job_status":"start","queue_duration_s":0.188915,"scheduling_latency_s":0.188928} gitlab # [ 286.887652] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.227Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"1c0dd5f669c9a9da9f87e056","created_at":"2026-09-06T16:10:07.022Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.023Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1c0dd5f669c9a9da9f87e056: done: 0.015554 sec","job_status":"done","queue_duration_s":0.188915,"scheduling_latency_s":0.188928,"redis_calls":7,"redis_duration_s":0.000941,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015554,"completed_at":"2026-09-06T16:10:07.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003644,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.938809] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.228Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"96ccb3cc9e8c66f21a2507eb","created_at":"2026-09-06T16:10:07.023Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.024Z","job_size_bytes":6,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-96ccb3cc9e8c66f21a2507eb: start","job_status":"start","queue_duration_s":0.20416,"scheduling_latency_s":0.204173} gitlab # [ 286.939405] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"96ccb3cc9e8c66f21a2507eb","created_at":"2026-09-06T16:10:07.023Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.024Z","job_size_bytes":6,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-96ccb3cc9e8c66f21a2507eb: done: 0.009813 sec","job_status":"done","queue_duration_s":0.20416,"scheduling_latency_s":0.204173,"redis_calls":7,"redis_duration_s":0.000801,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000232,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009813,"completed_at":"2026-09-06T16:10:07.238Z","load_balancing_strategy":"primary","db_duration_s":0.000314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.980384] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.239Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"052189ef7f8f6605d415e40c","created_at":"2026-09-06T16:10:07.024Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.026Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-052189ef7f8f6605d415e40c: start","job_status":"start","queue_duration_s":0.212855,"scheduling_latency_s":0.212867} gitlab # [ 286.981045] sidekiq-cluster[2774]: {"severity":"INFO","time":"2026-09-06T16:10:07.252Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"052189ef7f8f6605d415e40c","created_at":"2026-09-06T16:10:07.024Z","trace_propagation_headers":{"sentry-trace":"dedb46cc8e5b4ab4b3d62594b93357e1-96f2d52c5e024013","baggage":"sentry-trace_id=dedb46cc8e5b4ab4b3d62594b93357e1,sentry-release=Unknown"},"correlation_id":"01M1VQQVHBPX97S46FMHDSQHBC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-06T16:10:07.026Z","job_size_bytes":3,"pid":2774,"sidekiq_tid":"m83m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-052189ef7f8f6605d415e40c: done: 0.013399 sec","job_status":"done","queue_duration_s":0.212855,"scheduling_latency_s":0.212867,"redis_calls":7,"redis_duration_s":0.0008730000000000001,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013399,"completed_at":"2026-09-06T16:10:07.252Z","load_balancing_strategy":"primary","db_duration_s":0.001837,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.072953] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/09/06:16:10:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 635 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f684670ecafb28f8f83038a891201ff920ef7429/statuses, in 0.69 seconds) (finished: subtest: eval and build post commit statuses, in 6.38 seconds) (finished: run the VM test script, in 352.41 seconds) test script finished in 352.44s cleanup kill QemuMachine (pid 46) gitlab # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.56 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-06T16:08:58.572Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'f684670ecafb28f8f83038a891201ff920ef7429', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-06T16:09:59.352368Z', 'started_at': '2026-09-06T16:09:59.727270Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': 'f684670ecafb28f8f83038a891201ff920ef7429', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:09:59.706Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'f684670ecafb28f8f83038a891201ff920ef7429', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:09:59.706Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'f684670ecafb28f8f83038a891201ff920ef7429', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:09:59.706Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'f684670ecafb28f8f83038a891201ff920ef7429', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:09:59.706Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'f684670ecafb28f8f83038a891201ff920ef7429', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-09-06T16:09:59.706Z', 'started_at': None, 'finished_at': '2026-09-06T16:10:05.864Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'f684670ecafb28f8f83038a891201ff920ef7429', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-09-06T16:09:59.706Z', 'started_at': None, 'finished_at': '2026-09-06T16:10:05.864Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'f684670ecafb28f8f83038a891201ff920ef7429', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-06T16:10:06.011Z', 'started_at': None, 'finished_at': '2026-09-06T16:10:06.118Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]